Hi,
After I wrote about why JMeter is better than HP LoadRunner (or Performance Center) earlier this year, I want to share few services which only exists for who choose to use JMeter.
Aside of running load tests from your own servers either inside your Data Center or on Cloud hosting, which you can do with any load testing tool, by installing and manage these servers yourself, here is a list of services which allow you to get more, for less.
These (paid/free) services are waiting for you, if you are using JMeter:
After I wrote about why JMeter is better than HP LoadRunner (or Performance Center) earlier this year, I want to share few services which only exists for who choose to use JMeter.
Aside of running load tests from your own servers either inside your Data Center or on Cloud hosting, which you can do with any load testing tool, by installing and manage these servers yourself, here is a list of services which allow you to get more, for less.
These (paid/free) services are waiting for you, if you are using JMeter:
- BlazeMeter - don't mess around with building and managing your load testing servers. You can use this service to upload your existing JMeter scripts and resources (csv dataset files, etc) and run them from as many servers as you want, you pay for amount of servers/users you run over time and you can choose from which AWS locations to run the load test from. This service also provides nice central reporting capabilities.
- Gridinit - a new open source cloud load testing platform, which let you once again, utilize JMeter to run load tests from the cloud. The nice thing about this service is that you don't have to pay for anything, you can get the platform and install it on your own servers, either in the cloud or locally, and start load testing with nice web UI and some online reporting capabilities. You may also use this as a paid service and have them start needed servers in the cloud for you. Another nice feature is that you can do mixed mode load testing and utilize at once both their cloud service with combination of your own servers.
- Beatsoo! - a new free JMeter based application performance monitoring service. You can drop your existing JMeter scripts over there and start getting response times monitoring for your website (or any other thing your JMeter script interacts with) from world wide locations. Think how useful this is - you spend some time for load testing and improving system performance but why throwing away those scripts? Here you could re-use them for so-called "Synthetic User Monitoring" or End User Monitoring. This service will run your JMeter scripts from few world wide locations in very short intervals (around a minute or so) and allow you to compare your system performance over time and between different locations.
So here are more reasons for choosing JMeter over other load testing tools. Can you easily get you LoadRunner load tests run from several worldwide locations? Can you re-use your scripts for performance monitoring from worldwide locations?
Well, the truth about HP is that they have kinda LoadRunner SaaS but it seems like they don't really want people to use it, as it is very expensive and it is basically a hosted Performance Center, rather than Load Testing Saas.
For re-using of the scripts for monitoring, HP have the EUM BPM but it is not available as a service neither.
Perform better, use JMeter!
Well, the truth about HP is that they have kinda LoadRunner SaaS but it seems like they don't really want people to use it, as it is very expensive and it is basically a hosted Performance Center, rather than Load Testing Saas.
For re-using of the scripts for monitoring, HP have the EUM BPM but it is not available as a service neither.
Perform better, use JMeter!
Hi Shmuel, I am a beginner to Performance testing.I have got myself a simple 2 tier architecture to begin load testing via J-meter(.. the trial run was successful with load being transferred to the server side).Can you suggest me as to how different is performance testing by j meter on a local LAN setup from cloud based services(..Since finally i would be required to test the application on Cloud) also please tell me whether i can use j meter alone to test cloud applications or do i need to use cloud based services compatible with j meter.
ReplyDeleteHoping for a reply soon!!
Without going into too much of thinking, generally these are the diffs between local LAN load test and WAN load tests:
ReplyDelete1. One of the keys in performance testing is that you have a clean and un-changed environment between tests, so you can compare tests with specific changes that you've made (like comparing a load test with X traffic and then with 2X traffic). On LAN you should control easier the changes in the testing environment, as they are all locally to your lab, while on Cloud/WAN there are internet variables which you cannot control like others utilizing the network at the cloud provider side, or other network issues may appear in the way.
2. Cloud traffic comes from specific Data Centers around the world, if your service should be available only to the Indian market, what's the point to test it from outside of India network?
3. The total throughput or bandwidth you can support when traffic comes from the internet into your data center may be limited, thus load tests will show bottleneck which is external to your application. On the other hand, if you have a bottleneck with your ISP connection and you plan to provide this service over the internet, than it's good to find it during load tests rather than after go-live.
4. Intranet applications - cannot be tested usually from the internet, usually due to security considerations.
5. Analysis of results - because you cannot have exactly the same lab environment from test to test, results may be harder to analyze. On the other hand - the cloud providers mentioned above provide somewhat easier and cooler reports than are provided with JMeter, thus they allow somewhat easier analysis and overview of the results.
6. Costs of cloud service/servers vs. local machines.
Hey Schmuel, that was sooner than i expected,but any ways thanks much for the reply.I am only a manual tester (2 yrs exp.)who have been given the task of setting up performance testing environment in my company.The product is in Beta phase and they want me to move faster than everything.I will try to keep you posted with my progress also will try to get to you in case of any roadblocks.
DeleteHi Shmuel-
ReplyDeleteThanks for your summation of BlazeMeter's load testing platform. We value your feedback.
I also wanted to add that:
1- We can easily load test websites and apps, mobile sites and mobile apps as well as databases.
and
2-We have a number of plugins and integrations for even further simplified load and performance testing-a free plugin to JMeter, a Drupal module, a WordPress plugin, a Jenkins CI plugin and full integrations with TeamCity (JetBrains), CloudBees and New Relic (APM) and more coming up the pipeline.
Please keep up with us @ http://twitter.com/blazemeter
Best,
Daniela
Marketing Manager, BlazeMeter
http://blazemeter.com
Hi Shmuel, thank you very much for the useful information. Actually I am a developer and i have very limited knowledge on testing. But now I have a urgent task to load test a PHP web application. I am using JMeter and I am able to run some test cases, tested some URL's with varying concurrent users. Got results from JMeter in terms of tabular as well as graphical formats. But the main problem for me is I am not able analyse the test resuts (graph results) of JMeter. I google a lot for it. I want to know how to clearly analyse the test results and how to determine bottleneck conditions fro them. Basically I am not clearly understanding relations of the measurement units (throghput, median, avg. response time) used in the graphs. Any help from you will be greatly appreciated.
ReplyDeleteHi Yogesh,
DeleteIndeed analysis of load testing results is not always easy.
To be honest I've never found the benefit of using this listener... So I am clueless as you are.
You can direct this exact question to the JMeter users' mailing list at: http://jmeter.apache.org/mail.html
Good luck.
Hi Shmuel,
DeleteThanks for replying. I am using Summary report listener, which i find bit easy to analyze. Can you please briefly guide me on creating final analysis reports of load testing by giving some examples. I want to know which are the most important components of jMeter which are used in load testing. I installed google jMeter plugins and used Response Time vs Threads, Response Time over Time listeners. I can see increase in response time as load increases. But I am facing problems in exactly analyzing the data and create a analysis report form that data. Any example report will be much helpful. As I am a developer I have not done these things before, but I have been assigned this task now. So any help is really appreciated. Thanking you once again.
Once again, give the users mailing list a try and I promise you'll get an answer. This is not related to this post in any way...
DeleteGreat post, just a quick question. If one is already using JMeter and needs more advanced capabilities then it is obvious why they would use something like BlazeMeter, but is BlazeMeter's compatibility with JMeter scripts the only thing that differentiates it from competitors? Can't I also use Soasta or Blitz for this kind of test?
ReplyDeleteGood question.
DeleteShort answer: I think so, but...
If one is starting with load testing today and looking for load testing service there are quite few competitors.
I must be honest and say that I have never used soasta and blitz.io for more than getting familiar with them.
The fact that a service is based on JMeter is, in my opinion, a major advantage. There are many benefits when choosing JMeter based load testing over the others:
1. Portability: you may start today with blazemeter, switch tomorrow to gridinit(another jmeter based service).
2. Open technology: other services are using using propriatry and closed technologies. You can never know who will buy them or if they gonna close in two weeks from today.
3. Support and community: Less users and support is available for those services in compare to jmeter.
4. Maturity: JMeter is around for about ten years and still dramatically evolving. Strange cases and special needs are probably already taken care of in jmeter while on the new services you may need to ask and wait for the service provider to improve the platform/technology to support your need.
In my opinion the technology of load tests haven't changed over the last 20 years. Since early 90's Mercury LoadRunner and followers are basing on the same idea: simulating HTTP traffic.
I doubt any newcomers which do not change the rules of the game. And so far they don't so no need to play around with niche player which will not stay for long/get bought by huge corporate and fade.
This comment has been removed by the author.
ReplyDeleteyou can check the https://loadfocus.com JMeter Load and Performance service in the cloud on AWS, it's reliable and supports thousands of concurrent users
ReplyDeleteGood explanation on Performance testing. Keep posting :)
ReplyDeleteNot to mention the new comers, Jellly, which are not only providing JMeter test execution and analysis but also Cloud-based JMeter virtual user design.
ReplyDelete