Download Java JRE/JDK 7 from shell

A bit off-topic from this blog prespective but still wanted to share this as I couldn't find an answer to this.

I've been trying to install JRE 7 on my amazon ec2 server.
The first step to do so, is to wget (download) the rpm / bin install file, right?

Well it seems like since Java7, you must accept license agreement on the download page, before you can download java, which make sense. The problem with that is that you cannot accept this license agreement if you are on shell, doing wget.

[root@X opt]# wget http://download.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm
--2012-03-31 10:13:47--  http://download.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm
Resolving download.oracle.com... 67.148.147.41, 67.148.147.57
Connecting to download.oracle.com|67.148.147.41|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm [following]
--2012-03-31 10:13:47--  https://edelivery.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm
Resolving edelivery.oracle.com... 23.1.46.174
Connecting to edelivery.oracle.com|23.1.46.174|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://download.oracle.com/errors/download-fail-1505220.html [following]
--2012-03-31 10:13:47--  http://download.oracle.com/errors/download-fail-1505220.html
Connecting to download.oracle.com|67.148.147.41|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5307 (5.2K) [text/html]
Saving to: גdownload-fail-1505220.html.1ג
100%[==============================================================================================================================>] 5,307       --.-K/s   in 0.001s
2012-03-31 10:13:47 (3.40 MB/s) - גdownload-fail-1505220.html.1ג


So one way to work-around this is to download the file from a browser and copy over to the linux box, which for me, uploading 20MB file to amazon will take about half an hour or so..

The option I find to work around better for me is to run any network sniffer / IE developer tools / Chrome dev tools / etc and look for the final link which has the AuthParam=*.

So all you have to do now, is copy and paste this URL into your shell window like this:

[root@X opt]# wget http://download.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm?AuthParam=1333189254_3bd4c119fa6d0056c3661e50ebc8c521
--2012-03-31 10:19:17--  http://download.oracle.com/otn-pub/java/jdk/7u3-b04/jre-7u3-linux-x64.rpm?AuthParam=1333189254_3bd4c119fa6d0056c3661e50ebc8c521
Resolving download.oracle.com... 67.148.147.57, 67.148.147.41
Connecting to download.oracle.com|67.148.147.57|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21283819 (20M) [application/x-redhat-package-manager]
Saving to: גjre-7u3-linux-x64.rpm?AuthParam=1333189254_3bd4c119fa6d0056c3661e50ebc8c521ג
100%[==============================================================================================================================>] 21,283,819  16.9M/s   in 1.2s
2012-03-31 10:19:18 (16.9 MB/s) - גjre-7u3-linux-x64.rpm?AuthParam=1333189254_3bd4c119fa6d0056c3661e50ebc8c521ג

Good luck!

As I see many people still coming for this topic, please use the better solution described here: http://ivan-site.com/2012/05/download-oracle-java-jre-jdk-using-a-script/

Why Apache JMeter is better than HP Load Runner?

Few months ago I had a session with a friend of mine on performance testing and discussed the future load testing projects in his organization and on-going performance issues with current deployed systems.
During this session he shared with me his plans to invest on upgrading the current HP Performance Center 9.52 (Enterprise style Load Runner) to the new HP QC-PC 11 Platform during 2012.

It was after about a year or so I was familiar with JMeter and I was really pushing him to try it instead of Load Runner, as I found it much more reliable to work with.
I was trying to convey that instead of investing on licenses he can invest on better HR, better performance engineers.

So after some comparisons we made during this session, we got to a point where any lacking capabilities of JMeter in compare to Load Runner, is something that he can work around or live with.
The bottom line of that session was that there is nothing in Load Runner that he currently can't get out of JMeter (with some customizations and Plugins and aside of other protocols than HTTP/s).

Then he raised a question I never heard before:

What can JMeter give me, which Load Runner cannot?
I was always thinking the opposite way, always trying to compare with the leading product, with the "mature" product which is Load Runner.
I never thought that JMeter can provide something better than Load Runner other than it's being Free and Open Source.

So having this question echoing my mind for few months, I just had an enlightenment:

Community and support
JMeter has a strong supporting and prosperous community, questions asked or bugs found on the product got answers and fixes in a manner of minutes or hours (even on weekends!).
Bugs I found and reported got fixed in few hours and were publicly available on the next build (which is made on a daily basis or so).
With HP support I had a terrible experience, waiting hours for first response.
So whenever I found a bug in Load Runner I knew that if I will report it, I will probably have to wait for the next content pack or minor version for a few weeks in the good case (or usually forget about it and work-around it).
Even more than this – try googling for Load Runner errors / questions / topics and you find almost nothing that really helps you.

Scripting language
JMeter is java based, mostly you don't need to code anything, you only see a (gray) swing-based application and you easily play with test items which control the load test.

Load Runner is C based and you need to learn a propriety functions to generate the desired script.
Now who develops a web application in C? it is much easier to J2EE and even .Net developers (OO languages) to extend or work with a Java based application.

Simplicity
JMeter shows you everything you need in order to control the HTTP flow of the desired scenario, this way you know exactly what you are asking for and you know what response to expect.

Load Runner is hiding a lot in order to have a cleaner script code, this driving you crazy when you need to understand why certain things are working in a different way than what you expect.
Even with the new Ajax supporting engine "TruClient" they are doing the same trick – they hide things in these script "Levels", in order to make the script look nicer and you - a mad performance engineer who doesn't understand why the script behaves as it behaves.

Stupid-prof
JMeter force it's user to understand how web applications works, what is HTTP Request, what are headers, how a browser behaves and so on.

Load Runner is for everyone, so you find yourself trying to understand what is this load test engineer is talking about because he doesn't know what he is talking about.
I saw too many times load tests being executed by people who doesn't know the difference between a cookie and a header, or what is java script and where it being executed.
Do you want someone like this being responsible for you application performance or load stability? I wouldn't.


Conclusion
Of course JMeter has drawbacks which are fully documented on the net when comparing jmeter and load runner.
My point is that you can invest your budgets on utilities and you should, but you must invest some time in comparing the free utility with the expensive one.

In this case, if you are performance testing web based applications, JMeter may be better option for you.

Continue reading

Go to a following post on free jmeter-based cloud services.

Applications Monitoring is a Waste of Time

Organizations are making efforts on monitoring their information systems. These efforts are basically wasting the organization’s time and money.

There are great enterprise solutions from CA, IBM and others but even the leader (according to Gartner) which is HP BSM and its complementary components will fail in the basic purpose of monitoring. Not because these products monitor badly or incorrectly, but because organizations don’t understand the information they provide.

Yes you have a consolidate view of the system metrics for the relevant infra of the application, yes you do have some extra metrics from the Web Server, Application Server and DB Server.

You don’t understand which of those tens or hundreds of metrics are responsible for a failure in a case it happens and it will. There is no application with zero failures. Yes these products can analyze the historical metrics and alert when they get out of line, but who said that the historical metrics are good?

The answer in my opinion is Load and Performance Testing in order to get the metrics affect on the application performance and availability. Only when setting the relevant metrics on our monitoring systems we can fully take advantage of our investment in the monitoring solution.

I will try to convey in the following posts.

dynaTrace Browser Cache False Positive?

I am using dynaTrace Free Edition (AJAX Edition) for couple of years now as a Browser Side profiling tool and I am really pleased with it, especially with the new features found on version 2.x.

I was working on a performance optimization project for an Intranet web application couple of weeks ago and of course dynaTrace is in my toolkit. After working with the customer and understanding he's requirements I started the analysis.

The first thing I do is working with dynaTrace to get a performance overview. The overall rank was good but I got FAIL on the caching rank. Indeed no caching headers were set on the HTTP response headers on none of the cacheable resources.

Long story short, one of the recommendations in my final performance report was to use caching headers and I also declared that according to dynaTrace, setting these headers will save about 1.5MB of network traffic and about 10 seconds (on slow network connections from remote locations in the customer's so called "LAN").

This is where the first part of the story ends, with a report recommends on putting the main effort on this no caching problem.

This week I am instructing on a Performance course for programmers and I introduced them with dynaTrace as my favorite client side / browser side profiler. I love showing examples the freestyle way and asked one of the participants to donate his web application for the science. After browsing to the homepage of the web site, I closed IE and got back to the performance report. I noticed that on his web site he got FAIL on caching rank as well and he was amazed that there was no caching.

On one of the breaks the participant asked to investigate the caching problem. We've opened the report and got found that many flash (swf), gifs, css, js files were not cached. dynaTrace says that it sums up to about 500KB of transfer size and about three seconds in download time. The participant asked to reload the page again because it can't be true, the entire load time should be less the three seconds according to his experience. I reloaded the page and indeed the reload time was faster than the first request. We've re-opened IE and try again and the load time was fast again. It seems like the browser is using a cache but on dynaTrace it keep on showing the same recommendation – "Specifying Expires Headers can save up to 500KB in transfer and up to 3 seconds in download time".

We opened the Temporary Internet Files folder and found the cacheable files which were "not cached" in it. IE is saving any file on a local cache folder even if no cache headers were provided by the web server. This is true since IE 5 (http://msdn.microsoft.com/en-us/library/ms533020(v=vs.85).aspx#Leverage_the_HTTP_Expires_Header) and true to other browsers as well. This means that the load time of these components is faster than what prompted by dynaTrace.

When clicking on details on the resource we get to see the relevant HTTP request header and HTTP response header. In the response header we see:

1. HTTP/1.1 200 OK – which means we got a new file from the web server.

2. Content-Length 146471 bytes – which is the size of the response.

This is can't be true because we know that IE is using the files from the Temporary Internet Files folder – we see this component really fast on the browser. How can it be that dynaTrace shows a 200 OK ?

At this point I launched network sniffing tool – WireShark to see what's really going on and while working with dynaTrace in parallel I got:
1. The actual response code is not 200 OK but 304 Not Modified.

2. The actual size of the response body is not 146471 but 0 zero(no body).

So to sum things up this is what I learned:

1. Browsers will cache any resource (default configuration) even if no caching headers were provided.

2. Browsers will ask the web server if modified since on any resources on the caching folder which has no caching header (you can see it in the screen cap, the red line starting with If-None-Match).

3. Web servers have a special response when the browser already has an up-to-date resource – 304 Not Modified.

4. dynaTrace 2.1 has a bug – showing wrong information about this kind of requests.

Last thing - dynaTrace prompt for this caching problem and this is correct even with this bug. We still waste the browser's connections on asking for validation of these resources and on high latency networks this is a waste of time. If caching headers were provided, the browser wouldn't even ask to validate those resources.



J2EE Application Server Windows Vs. Linux Benchmark

Old debt of mine since my last post almost two years ago while comparing performance of Oracle Application Server, which is a J2EE Server, running on Windows and on Linux.

I won't post the full document which is about 30 pages but I will summarize things as OAS / IAS is deprecated after Oracle acquired BEA WebLogic Application Server.

To the point - we have build a Java simple application doing few things:

1. I/O test:
   Reading a file located on the system disk (server side) and displaying its contents to the client.
2. DB test:
   Executing query to an Oracle DB and displaying the result to the client.
3. HTML test:
   Simple HTML page containing some text an few images in different sizes.
4. Memory Leak test:
   Memory leak, by adding objects to the session and not releasing them.

We used HP's Load Runner and run some users doing each of the above actions, each action in a separate test.

At first we found that Windows and Linux (JVM) crushed and fail-back in the exact same manner when dealing with the Memory leak scenario.
The same with the queries to Oracle DB scenario (bottle-neck was with [UPDATE LATER]).

HTML test was the first time we finally got different results.
After adjusting the Apache.conf few times we got an amazing result,
Windows-based server couldn't keep up with over 300 concurrent users (bottle-neck was CPU) while in the exact same scenario Linux-based server got 1000 concurrent users (and CPU was less then 75%) !

Encouraged with our last results we continue to the last test, I/O test.
We found that Linux can handle requests much faster when parsing the exact same file with the same amount of concurrent users.
[UPDATE LATER]...

Our final conclusion is that J2EE performance is far better on Linux env rather than on Winodws env. It is extremely obvious with the HTML test (which is actually not J2EE) that Apache on Linux can handle four-times throughput than on Windows.

Feel free to ask for more information if you feel something is missing.

Oracle Application Server (OAS) - on Windows or Linux?

There is a buzz over the last years about which operating system is more efficient as a web server platform. We are using OAS as our web and application server (which based on Apache as web server) and nowadays did a research to compare Windows 2003 Enterprise R2 vs Red Hat Enterprise Linux 4 and got a really promising winner...

I will post the results soon but for now you can go visit this web site which did a similar benchmark and got some really confusing results - which we got as well at first, but after some research and learning were fixed.

SSO for Oracle Grid Control

Although there is a lot happening, it seems like I'm almost never pick myself together to write a post over here.

This time I'll share a solution I made up to create a Single Sign-On module for Oracle Grid Control. Well I assume you already know what Grid Control is all about but if you don't you can check it out on Oracle's web site.

Something like a year ago I first heard about this product and me and some of the people I'm working with agreed immediately that we should do a POC of it in our network... Few weeks later we had it installed and agents were deployed on over 50 servers. Out of few minor problems as with any other product we integrated into our IT environment it was working and pretty much kept its word.

A while has passed until we actually put it on our production environment and now we use the latest version (Grid Control R3).

All this time there was a very annoying process in this product – as you guessed – the logon screen that shows up each time you want to use the application.
I picked up the phone and asked Oracle support team if they have any SSO solution for Grid Control because there is no logic in asking a user his credentials if he already authenticated against a domain. Oracle's local support team said that there is no SSO solution and when I asked them why, they said that this product is supposed to serve a small group of admin users and because of that there is no excuse to create a SSO module for it. This is where I come into the picture and in the next paragraphs I will explain how I built SSO module for Oracle Grid Control.

Well the Management Server is a normal J2EE application running on a standard Oracle Application Server and as any J2EE application we could relative easily change it. The Idea was to add a MOD_NTLM to the HTTP Server (OAS is based on Apache) and change the logon module to use the remote user id and log him in with the right permissions.

So I'll start from the bottom – DB.
SSO in its basic level is all about logging someone with his domain authentication to our application and giving him the right permissions. This means that we need two tables:
First is a table with our local application users(id, username and password).
Second is a table which maps between a domain user to a local user/role(domain username, local user id from the first table).

On top of it we should create a simple page which take the remote username ,ask the DB which local user should he be logged on with, put this values into the form and add some java script that submits the logon form for the user.

Well this is a problem because I was counting that the logon page is some kind of a standard technology(JSP or servlet) but…
The logon screen is a .uix page (Oracle's tag based server side pages) called logon.uix located in the OC4J_EM directory with all other application files… There is no way I will learn another language for this I told myself and so I came up with the next idea:
I'll put some java script code in the uix (there is a place to put raw html in those pages) logon screen that opens up a http request to another page that is filtered with MOD_NTLM which will do the SSO logic (select the right user from our tables). This second page will be a JSP that will write back a XML response with this information and back to the uix java script that will parse it and put it into the username and password fields and submit the form (yeah you are right, this is known as AJAX).

This is it. SSO is actually very easy to create.
'Till next time,
Good luck!

*You should put some encryption on the first table because saving DB passwords in clear text is not recommended.

*I never tried it but this solution should work fine to be a SSO for Oracle Enterprise Manager (OMS) for any Oracle Forms Server, Oracle Discoverer or any other Oracle Application Server(R3 and above).

IIS based Web Service - Delayed Response

After a long break updating my blog, I'm back.
I had some interesting issues in the last few month and I hope I'll have the time posting them here....

I'll begin with the oldest one.
Few months back one of the developers in the development team was complaining about a very slow response times from a web service he developed and was running in our Test environment.
He told me that the problem only occurs in the first request, or on the first request after long time (30 mins or above), but he cannot be exact and can't reproduce it.
The normal response time is about 1-2 seconds but to the first one, which takes almost 30!
He also told me that there is nothing 'heavy' or complex in the initialization process and he suspects that the problem is in IIS or the CLR...

I followed his directions and the first thing came in my mind is the "Idle timeout" that can be configured to IIS Application Pools. Surprisingly it was actually configured to 20 mins! I turned it off (to never shutdown an idle application pool).

After couple of hours he came back complaining that the problem still occurs.
My next step was to look for some ASP.NET configuration options in web.config and machine.config but I found nothing really related...

I had no direction and the only thing in my mind was that maybe something is wrong with the server(IIS or .NET Framewok), so I set up a virtual server with other OS version(Wndows 2003 Web Edition) and tried causing the problem to appear again.

I noticed that the only way to reproduce the behavior was to boot the server - Killing the worker process or restarting the IIS didn't trigger it.

Now with the option to reproduce in my hands, I could really start digging. I started (Microsoft's) System Internals' Process Explorer and watched the w3wp.exe of the WebService (w3wp.exe is the instance of the "Application Pool" you see in IIS). I checked to see if it was working and using CPU and not just waiting or hang on something. I drilled down to its threads tring to detect where the problem is. If the problem was in his code I could see it, but it was happening very long time before his code was invoked. I saw some threads waiting on a method named CompareAssemblyIdentity (a method used to "compares two assembly identities to determine whether they are equivalent"). Few moments later I noticed some new child processes running under the worker process, named csc.exe (C# Compiler) and they (in their turn) created another process as well !! Only then I realized the problem! ASP.NET is Recompiling the classes for the first time the Web Service is used! It is that simple.

Boot is not the only trigger for ASP.NET recompilation and it wasn't the case of the poor developer as well. The other trigger is much more relevant to his case - changes in the code. He just forget telling me he was still working on it and publishing his code relative frequently to the test server...

Kernel Memory leaks - Part 2

After a while not able to reproduce the case I've managed to get it again and copy all the data from Poolmon. The total of Paged-pool shown in the upper-right corner says 207MB but when summing each allocation for each tag (only Paged of curse) I got 270MB !!

Still have no answer.

Kernel Memory Leaks

For the last couple of years we are getting alot of servers' hangs ,due to both Paged and Non-Paged Pool depleted. I've managed to trace the leaking applications and either it an application we wrote or a third party application - closing it (killing it's process) just frees part of the leaked handles.

When monitoring with poolmon I saw the handles getting freed but not as they should have been.
Lets say that the total memory those handles were taking is 100MB in paged pool and the total use of paged pool was 150MB. Killing the process that created them should free them all right? - Wrong. While killing the process freed 90MB of the handles, the total should have dropped to 60MB, but its not! There seems to be unlisted handles either by purpose or by mistake/bug of the OS.

I intend to get to the bottom of this.