Rational Performance Tester

I just received an evaluation copy of the new version of Rational Performance Tester. IBM Rational have re-written the tool from scratch and based it on the open-source Eclipse framework. The tool has only just come out of beta testing and currently supports web protocols only. Everyone in the industry seems to be keen to know how it stacks up against the competition.

First off, my impressions of the old version…

Coming from a LoadRunner background, my opinions on the tool were a little mixed. Performance Tester was easy to learn and easy to use but it is definitely missing some features when compared to LoadRunner and, possibly because of my background, the way the tool did some things just felt wrong. It must also be noted that I have only used the web protocol. Other protocols may behave differently.

In my opinion, the downsides were

• It uses its own custom scripting language. Once you know how to program in one language, it’s easy enough to pick up another (as long as it doesn’t use a totally different paradigm, like functional programming), but it all feels a little unnecessary.
• Manual correlation is done using regular expressions. So sites like the Regular Expression Library are useful.
• The tool automatically correlates everything. This makes the scripts ugly. This is great if your script happens to work for all the data you are going to use with it. It is painful to debug if it doesn’t work sometimes. This is why a Rational tool expert will tell you it is easier to re-record a script than edit it. This is also why Rational users don’t generally put text checks in their pages.
• When you are running a test, you have nowhere near the visibility into what is going on that LoadRunner gives you.
• I found the way it handled chunked encoding really weird.
• Connection handling is not transparent to the user.

But on the plus-side…

• The automatic correlation works most of the time. This can be a big time saver.
• Their support is excellent.
• It is significantly cheaper than LoadRunner.
• The tool gives you the option of using a persistent cursor for your data tables. So if you had a test case that, for instance, deleted a user from a database, you would not have to restore the database or re-add users between test runs, you could just continue from the same point in your list of users. This is a feature that LoadRunner is missing.

I will post a review of the new version of IBM Rational Performance Tester once I have had a chance to play with it for a while.