My Load Test

A LoadRunner feature that has made my life a lot easier has been ContentCheck rules, which are available in the script runtime settings. If you are using a web-based vuser type, you can configure your LoadRunner script to search through all returned pages for strings that match known error messages.

Using web_reg_find functions is fine, but when you get an error LoadRunner reports it as “failed to find text” instead of something more descriptive.

I will always create rules for any error messages I find during scripting and, if I receive an error while running a scenario, I will add the error message from the snapshot in the scenario results directory (the snapshot on error feature is very useful).

All this is pretty obvious if you have taken the time to explore LoadRunner’s features or you have attended a Mercury training session, but I recommend taking things a step further.

• Ask your developers for a list of all the error messages that the application can throw. This should be easy for them to provide if the application is well designed and stores all the message in some kind of message repository instead of sprinkling them throughout the source code.
• Include error message for functional errors that you are likely to encounter. Creating a rule for “incorrect username or password” may save someone 20 minutes of investigation when they first run the script after the database has been refreshed.

If you prefer to have error message you are checking for in the script (where you can add comments to them) instead of the runtime settings, you can use the web_global_verification function instead. The only difference between the two is the error message that LoadRunner will include in its log:

Action.c(737): Error -26368: “Text=A runtime error occurred” found for web_global_verification (“ARuntimeErrorOccurred”) (count=1), Snapshot Info [MSH 0 21]

…compared to:

Action.c(737): Error -26372: ContentCheck Rule “ARuntimeErrorOccurred” in Application “Webshop” triggered. Text “A runtime error occurred” matched (count=1), Snapshot Info [MSH 0 21]

And finally, ContentCheck rules can be easily exported and shared between scripts, which can be a nice time-saver.

A common question on the LoadRunner support forums is “I’ve got an application that uses this new thing called AJAX. Will LoadRunner work with my application?” …and the answer is “yes…but…”.

The “but” is a big one . An “AJAX” application can mean anything from a simple webpage that updates some fields with information from the server (without reloading the whole page), to a horribly complicated rich GUI interface created with JavaScript.

Of course LoadRunner is going to be able to handle anything that is sent over HTTP, but scripting might not be the usual simple Record-Correlate-Playback, and the chances of the new Click and Script vuser type working are much lower than with a standard web application.

As a quick introduction to AJAX I have prepared an exercise using the simplest AJAX application that I could find.

Google Suggest is just like Google’s regular search interface, except that every time you type a letter in the search field, a request is made to the server that returns search suggestions.

The exercises are as follows:

1. Become familiar with the application – both at an end-user level, and at the HTTP level. Record the script in HTML and URL mode, then try the Click and Script vuser type (at the time of writing, this vuser type will not correctly record the application).
2. Create a script for this application. It should accept a parameter file that contains a valid search term on each line. Create the following transactions load_search_page, enter_character (called every time a character is entered in the search field), and final_search (where the Google Search button is pressed).
3. Imagine that you’ve noticed an odd step-like pattern in the Percentile graph in LoadRunner analysis. You hypothesise that response times differ depending on how many characters have been entered in the search field. Modify the enter_character transaction name to include the length of the search term every time it is called.

Treat this with the same level of care that you would give a script that you were creating for a paying client.

I have added some notes and a solution script to the comments section of this post. If you are going to attempt this exercise, please do so before reading the comments.

There’s a saying in software circles that “writing the first 90 percent of a computer program takes 90 percent of the time… The remaining ten percent also takes 90 percent of the time.” A year ago, Mercury started telling people about their new GUI level virtual users for the web. Their promise was an impressive one – no more correlation. Anyone who has scripted for the web knows that correlating a web script can be either incredibly easy or painfully difficult if, for example, the web application constructs HTTP POSTs with an unholy mess of JavaScript. Mercury was promising to make this unnecessary by creating a playback engine that understood what the web page was doing just as well as your web brower, allowing you to create scripts that were all about clicking on buttons and links in a browser rather than sending HTTP requests to the web server.

Presumably, getting the remaining ten percent of this functionality working perfectly before the release of LoadRunner 8.1 was too difficult, so the software was shipped with this Vuser type disabled. The good news is that if you have a copy of LoadRunner (or Performance Center) 8.1, you can enable this vuser type yourself and have a play before it is officially released.

To enable this functionality do the following…

1. Open the \dat\protocols\webjs.lrp file under your LoadRunner or Vugen installation directory.
2. It is in standard Windows ini format. Find the [Protocol] section and change the line that says “Hidden=1″ to “Hidden=0″ .
3. Now open Vugen and create a new script. You will see that there is a new option to create a Web (GUI level) Virtual User

Rumour has it that the GUI level web vusers will be officially released by Mercury in 4-6 weeks with the release of service pack 2 for LoadRunner and Performance Center. This will mean that it will be officially supported by Mercury.

I do not normally recommend that users jump into brand new technology, but this vuser type offers such a huge advantage over regular HTML and URL-mode virtual users (not to mention every other web load testing tool from all other companies), that it is worth spending an afternoon seeing if it works with your application. If it doesn’t, you’ve lost an afternoon; if it does, you’ve saved weeks of your time.

A question that regularly comes up in the Mercury Support forum is how to get a LoadRunner script to handle websites that require you to type in the characters from a picture of a funny-looking piece of text before you can perform an action such as creating a new account. “Surely someone has come across this before, and has figured out how to correlate these values?”

Well…uh…no they haven’t. These are Captchas – they are used to prevent bots from doing nasty things like signing up for thousands of Hotmail accounts or leaving comment spam. If it was possible to write a LoadRunner script that could handle these automatically, then they wouldn’t be very effective.

Automated solutions used by spammers revolve around text recognition software and are inneffective. A spammer might be happy with a 0.01% success rate when signing up for webmail accounts, but it is hardly useful to a load tester who wants a 100% success rate. Boing Boing’s solution is also not very useful for load testing.

If you need to load test a website that uses captchas, then the only practical solution is for the developers to include the captcha value somewhere in the web page that is presented to the user. You can then correlate the value the same way you would normally. Just make sure that the code that does this is removed before the website goes live.

I don’t ever like to feel that I am wasting a client’s time by doing unnecessary load tests, but a long time ago I had an experience that altered my definition of what a necessary test is.

I had come to this project after the scope had been decided and the Detailed Test Plan had been written. The overall system was made up of four applications; privately I queried why we were bothering to test ApplicationX, which was an exceedingly low volume application.

ApplicationX was not business-critical or customer-facing. It was performing acceptably with a production-sized dataset and a small number of test users. Its functionality was unlikely to put significant load on the system, and there would be a maximum of 6 concurrent users in Production. Not load testing ApplicationX seemed like a perfect way to save some time.

Fortunately the Test Manager wanted to stick with the original plan (rather than having to get sign-off on an amended plan), because I was very very very wrong.

Here’s a graph of what happened as I slowly ramped up my virtual users…

I had never seen a web application fall over under a load of only two hits per second before. I didn’t think it was possible. My grandmother could handle two hits per second creating HTTP headers on a typewriter and sending the packets via carrier pigeon.

The worst part of this was that ApplicationX was running in the same JVM as an application that was business critical, so when it went down, it brought the other system down with it. A severity-3 defect suddenly became a severity-1 “oh my god, we can’t do business” defect.

So, the lesson I took away from that experience was that even if an application is not necessarily a candidate for load testing, but it shares infrastructure components with an application that is business-critical, then it is a good idea to consider at least some rudimentary sociability testing under load to shake out any really nasty problems like this one.