Few months ago I had a session with a friend of mine on performance testing and discussed the future load testing projects in his organization and on-going performance issues with current deployed systems.
During this session he shared with me his plans to invest on upgrading the current HP Performance Center 9.52 (Enterprise style Load Runner) to the new HP QC-PC 11 Platform during 2012.
It was after about a year or so I was familiar with JMeter and I was really pushing him to try it instead of Load Runner, as I found it much more reliable to work with.
I was trying to convey that instead of investing on licenses he can invest on better HR, better performance engineers.
So after some comparisons we made during this session, we got to a point where any lacking capabilities of JMeter in compare to Load Runner, is something that he can work around or live with.
The bottom line of that session was that there is nothing in Load Runner that he currently can't get out of JMeter (with some customizations and Plugins and aside of other protocols than HTTP/s).
Then he raised a question I never heard before:
What can JMeter give me, which Load Runner cannot?
I was always thinking the opposite way, always trying to compare with the leading product, with the "mature" product which is Load Runner.
I never thought that JMeter can provide something better than Load Runner other than it's being Free and Open Source.
So having this question echoing my mind for few months, I just had an enlightenment:
Community and support
JMeter has a strong supporting and prosperous community, questions asked or bugs found on the product got answers and fixes in a manner of minutes or hours (even on weekends!).
Bugs I found and reported got fixed in few hours and were publicly available on the next build (which is made on a daily basis or so).
With HP support I had a terrible experience, waiting hours for first response.
So whenever I found a bug in Load Runner I knew that if I will report it, I will probably have to wait for the next content pack or minor version for a few weeks in the good case (or usually forget about it and work-around it).
Even more than this – try googling for Load Runner errors / questions / topics and you find almost nothing that really helps you.
Scripting language
JMeter is java based, mostly you don't need to code anything, you only see a (gray) swing-based application and you easily play with test items which control the load test.
Load Runner is C based and you need to learn a propriety functions to generate the desired script.
Now who develops a web application in C? it is much easier to J2EE and even .Net developers (OO languages) to extend or work with a Java based application.
Simplicity
JMeter shows you everything you need in order to control the HTTP flow of the desired scenario, this way you know exactly what you are asking for and you know what response to expect.
Load Runner is hiding a lot in order to have a cleaner script code, this driving you crazy when you need to understand why certain things are working in a different way than what you expect.
Even with the new Ajax supporting engine "TruClient" they are doing the same trick – they hide things in these script "Levels", in order to make the script look nicer and you - a mad performance engineer who doesn't understand why the script behaves as it behaves.
Stupid-prof
JMeter force it's user to understand how web applications works, what is HTTP Request, what are headers, how a browser behaves and so on.
Load Runner is for everyone, so you find yourself trying to understand what is this load test engineer is talking about because he doesn't know what he is talking about.
I saw too many times load tests being executed by people who doesn't know the difference between a cookie and a header, or what is java script and where it being executed.
Do you want someone like this being responsible for you application performance or load stability? I wouldn't.
Conclusion
Of course JMeter has drawbacks which are fully documented on the net when comparing jmeter and load runner.
My point is that you can invest your budgets on utilities and you should, but you must invest some time in comparing the free utility with the expensive one.
In this case, if you are performance testing web based applications, JMeter may be better option for you.
Continue reading
Go to a following post on free jmeter-based cloud services.
