My Load Test

Recently a client asked me “What’s the difference between LoadRunner and Performance Center? They’re both sold by the same company, but they seem to do much the same thing.”

I think he could see that I was about to launch into a half-hour lecture on the topic because he quickly added “Just give me the elevator pitch”.

I thought for a second and explained “Well, obviously they are both used for performance testing, but Performance Center is the Enterprise version of LoadRunner; so it suits larger companies who need to run performance testing for multiple projects at the same time without the problem of sharing a Controller or purchasing duplicate licenses.”

He put his hand on my shoulder and said “That’s an okay pitch, but it has one big problem. Let’s go and get a coffee, and I’ll tell you what I think of when I hear someone say ‘enterprise software’.”
(more…)

Imagine that you need to load test a Network Management System (NMS), which might receive hundreds of SNMP traps every second from tens of thousands of devices on the network. Clearly it is impractical to actually deploy thousands of devices in a test environment, so you need a tool to generate the SNMP traps at the required volume.

A quick search for SNMP load testing tools shows that they are all fairly primitive, with poor support for creating SNMP traps with dynamic field values, and no support for integrated system monitoring. There are definitely no tools with the same level of functionality as LoadRunner.

Unfortunately LoadRunner does not have an SNMP vuser type…but fortunately LoadRunner is endlessly extensible, if you are not afraid to get your hands dirty. Read on for how I created a LoadRunner script to send SNMP traps to an NMS…
(more…)

If you need to run a Unix command on a remote server with a LoadRunner script (running on Windows), then there is a neat program called plink that you will find very useful.

If you use any variety of Unix, you will probably already by using PuTTY; well plink is is distributed with PuTTY, and allows you to run remote Unix commands from your DOS prompt.

In this example, I have used it to collect disk I/O metrics from a remote Solaris server.
(more…)

Everyone should already know that you can run a command-line program from a LoadRunner script using the system() function. If you have used this, you would already know that things become a little awkward when need to capture the output of the program.

The common solution is to pipe the output of the command to a file (i.e. system(“my_command > output_file.txt”)), and then read the contents of the file. This is clunky and awkward, and not safe when multiple virtual users are writing to the same file.

A much better solution is to run all your command-line (DOS) programs using the popen() function. Read on for examples…
(more…)

On increasingly rare occasions, you might be required to load test an application that has a Java-based client. While it’s nice for users to have an application that has a rich GUI, it will probably lead to a world of pain for you as a performance tester.

Don’t immediately reach for LoadRunner’s Java Record-Replay virtual user type. It might not be the right solution.

This article will guide you through your first steps when selecting the correct vuser type to test an application with a Java-based client…
(more…)

I like posting LoadRunner code snippets on My Load Test and, judging by the emails and comments that I get, a lot of people find them really useful. To make my code more readable, I have added syntax highlighting that uses the same colours you see in VuGen. If you would like to download a copy of the syntax highlighting code to use on your own blog or forum, or you would like to learn more about how I did it, read on…
(more…)

The Virtual Table Server is great for most situations where your virtual users need a common data pool, but the limitations of the API mean that it is a bad fit in some cases.

Say you need to find a row in the data table with a specific value. On a real database you could just do a simple SELECT statement. With VTS, you would have to write code to iterate through every row in the table and check the column value each time.

So sometimes you need to use a real database. Interfacing with a real database is not much harder than interfacing with VTS (as the attached script demonstrates). You will spend more time setting up the database than writing code.

Interfacing with a real database will also allow you talk to the database your application uses (only do read-only operations, and be careful of any additional overhead you might introduce during your load test).

This example users the Java vuser type, and interfaces with a MySQL database. The beauty of JDBC is that to port this code to another database just requires changing the line of code that specifies the database driver (assuming that you have not used database-specific SQL code).

On with the example…

Setting up the database

1. Download and install MySQL.
2. Create a database
   mysql> CREATE DATABASE loadtest;
mysql> SHOW DATABASES;
mysql> USE loadtest;
3. Create a table
   mysql> CREATE TABLE message (time_sent BIGINT UNSIGNED, identifier VARCHAR(127));
mysql> SHOW TABLES;
mysql> DESCRIBE message;

4. Load data into the table
   mysql> INSERT INTO message VALUES ('1178858071111','asdf1234');
mysql> INSERT INTO message VALUES ('1178858071112','asdf1235');
mysql> INSERT INTO message VALUES ('1178858071113','asdf1236');
mysql> INSERT INTO message VALUES ('1178858071114','asdf1237');
mysql> INSERT INTO message VALUES ('1178858071115','asdf1238');
mysql> SELECT * FROM message;
5. Create a user account with access to the database.
mysql> GRANT ALL PRIVILEGES ON loadtest.* TO 'loadrunner'@'%' IDENTIFIED BY 'loadrunner' WITH GRANT OPTION;
Note username/password is loadrunner/loadrunner.
mysql> SELECT * FROM mysql.user where User = 'loadrunner';

Verifying external connectivity to the database

1. Download an SQL query tool to verify that you can successfully connect to the database.
   I recommend that you check out SQLeonardo.
2. Download the JDBC driver for MySQL.
   MySQL now produce their own JDBC driver (Connector/J)
3. Configure the SQL query tool.
   nickyb (the creator of SQLeonardo says)…

   Suppose you have downloaded and unzipped sqleonardo into c:\sqleonardo and the mysql jdbc driver into c:\sqleonardo\mysql

   Run SQLeonardo and into the “metadata explorer” do:
   - choose the menu “actions>new driver…”
   - check “add library (browse filesystem)” and click “next >”
   - select the jar file into c:\sqleonardo\mysql and click “next >”
   - type into the textfield named “name:” => MySQL
   - select into the combobox named “driver:” => com.mysql.jdbc.Driver
   - into “example:” => jdbc:mysql://[host][:port]/[database]
   - click ok.
   now you have registered the driver!

   Select the item “MySQL” appeared into the tree and:
   - choose the menu “actions>new datasource…”
   - replace jdbc:mysql://[host][:port]/[database] => jdbc:mysql://localhost:3306/
   - put username and password and click ok.
   now you have added your database profile.

   Select the item under “MySQL”…you need now to test the connection!

   So our database connection string would be jdbc:mysql://www.myloadtest.com:3306/loadtest (assuming the database is installed on the default port).

4. Test the connection.
   Try running SELECT * FROM message; in your SQL query tool.

Writing the code

1. Java.sql javadocs are here. There are numerous tutorials on the web, and you have my sample code (text, zipped). What more do you need?

This is a quick walkthrough of a LoadRunner script I created to load test a “web” service that communicates with SOAP messages sent over JMS.

Note that this script supports the queuing/point-to-point model of JMS (rather than the publish and subscribe model), works with JMS version 1.1 rather than 1.02b (as it uses the BytesMessage.getBodyLength() method among others), and has been tested with Java 1.5 using the Tibco implementation of JMS.

The LoadRunner script can be downloaded here, the source code is available here, and a mirror of the JMS 1.1 specification can be downloaded here. People using Tibco’s JMS implementation can download their Javadocs here (10MB). Feedback is appreciated.

From looking at the script, you may wonder why I chose to write my own script using the Java vuser type instead of using the Web Services vuser (version 8.1, Feature Pack 4 supports web services using JMS as well as HTTP), but that is a story for some other time.

Anyway…for the tiny number of people that I haven’t lost already, here’s how it all works…

At a high level, we are trying to put a SOAP message on a queue (MyLoadTest.Warehouse.Request), receive the corresponding response from a different queue (MyLoadTest.Warehouse.Response), and check that the response message matches our expected message. Download my source code and follow along.

On with the detail…

The javax.jms.* classes provide interfaces only, other vendors have provided implementations of these interfaces (I am using Tibco’s implementation – found in com.tibco.tibjms.*). This should mean that any JMS code you write will be portable between implementations; whether this is true or not is something I haven’t tested.

Initially we perform a JNDI lookup to find a ConnectionFactory and get the details for the queues we will be using.

We then create a Connection and a Session, and then create the MessageProducer (using the request queue details) that we will be using to send our message to the request queue.

We create either BytesMessage or a TextMessage (MapMessage, ObjectMessage and StreamMessage have not been implemented in the script), and set the JMS header properties. The important JMS headers are the JMSDestination (the request queue), and the JMSReplyTo (the response queue). We can also set Message properties, which are simple name/value pairs. We then set the Message body, using a string constant containing the SOAP XML message.

We only want to measure system response times, so we only put our LoadRunner transaction timing points around the code that sends and receives the message.

Sending the message is as simple as calling myMessageProducer.send(myMessage).

In the following extract from the Tibco EMS logs, you can see the JMS headers, the Message properties and the Message body (shown here with a size only). The JMS headers show the request and response queues along with a unique msgID.

2007-02-26 13:20:07 [MSG:73422]
received from user='anonymous':
connID=1669
prodID=16906
msgID='ID:SVP-EMS-SERVER.391E45DA4ADA6C8:3083'
Time=1172456256000
mode=PERSISTENT
queue='MyLoadTest.Warehouse.Request'
msg=BytesMessage=
{
  Header=
  {
    JMSDestination={QUEUE:'MyLoadTest.Warehouse.Request'}
    JMSReplyTo={QUEUE:'MyLoadTest.Warehouse.Response'}
    JMSDeliveryMode={PERSISTENT}
    JMSPriority={4}
    JMSMessageID={ID:SVP-EMS-SERVER.391E45DA4ADA6C8:3083}
    JMSType={BytesMessage} JMSTimestamp={1172456256000}
  }
  Properties=
  {
    "SOAPJMS_requestIRI"={string:''}
    "SOAPJMS_targetService"={string:''}
    "SOAPJMS_soapAction"={string:''}
    "SOAPJMS_isFault"={string:'false'}
    "SOAPJMS_contentType"={string:''}
    "SOAPJMS_soapMEP"={string:'http://www.w3.org/2003/05/soap/mep/request-response/'}
    "SOAPJMS_bindingVersion"={string:''}
  }
  Body=
  {
    byte[]:1126 bytes
  }
}

To receive a Message, we must create a MessageConsumer (using the response queue). Rather than simply receiving the messages from the queue in the order they arrived (FIFO – which would mean that we could be receiving the wrong response message when we are running load against the system), we can use a message selector to make sure that we receive the message that we want from the queue.

In the log above, we saw that each JMS message has a unique JMSMessageID. When a response is sent by the server, the response message has a JMSCorrelationID that matches the JMSMessageID of the message it is replying to. A message selector is defined in the same way as the WHERE clause of an SQL statement, so our selector would be
"JMSCorrelationID = '" + myBytesMessage.getMessageID() + "'".
Note that the unique message ID is only set when the message is sent so, if you are using a selector, you must create your MessageConsumer *after* you have called myMessageProducer.send(myMessage). A selector can use any of the JMS headers and Message properties, but cannot use the Message body.

Using the MessageConsumer, receiving the correct message from the queue is done by simply calling myMessageConsumer.receive(timeout). The script will wait to receive the message until it reaches the timeout value. If no message is received before the timeout, the response message will be null. Remember to close the MessageConsumer, or you will leak and end your test with thousands of open receivers.

Here is the Tibco EMS log entry for the response message. You can see that the message has a different JMSMessageID, and has a JMSCorrelationID with the message ID of the request it is replying to.

2007-02-26 13:20:09 [MSG:73425]
received from user='anonymous':
connID=1714
prodID=19956
msgID='ID:SVP-EMS-SERVER.391E45DA4ADA6F5:7835'
Time=1172456409014
mode=PERSISTENT
queue='MyLoadTest.Warehouse.Response'
msg=BytesMessage=
{
  Header=
  {
    JMSDestination={QUEUE:'MyLoadTest.Warehouse.Response'}
    JMSDeliveryMode={PERSISTENT}
    JMSPriority={4}
    JMSMessageID={ID:SVP-EMS-SERVER.391E45DA4ADA6F5:7835}
    JMSCorrelationID={ID:SVP-EMS-SERVER.391E45DA4ADA6C8:3083}
    JMSTimestamp={1172456409014}
  }
  Properties=
  {
    "SOAPJMS_requestIRI"={string:''}
    "SOAPJMS_contentType"={string:'text/xml'}
    "SOAPJMS_soapMEP"={string:'http://www.w3.org/2003/05/soap/mep/request-response/'}
    "SOAPJMS_bindingVersion"={string:''}
  }
  Body=
  {
    byte[]:681 bytes
  }
}

Extracting contents of the Message body from the JMS Message is done by calling myMessage.getText() if it is a TextMessage (if it is a BytesMessage, it is a bit trickier – see the attached script).

And finally comparing the received SOAP message to the expected message is just done using String.equals().

Remember to close the Session and Connection at the end of the script.

The exercise of modifying the script to send many different messages to different queues (which would mean you only have to maintain one script instead of one script per SOAP operation) is left as an exercise for the reader…