My Load Test

As part of the LoadRunner 8.1 Hands-On Lab training, there is an exercise to enhance a VuGen script that books a flight using the Mercury Tours application.

Company “A” wants the script to generate 10 digit credit card numbers instead of supplying them from an external source on every iteration and for an unlimited number of iterations.

How do you make the script generate credit card numbers for every iteration?

As the Mercury Tours application does not check if the credit card numbers conform to a valid format, the answer to this question is just to replace the file-based credit card parameter with a random parameter using a number format of %010lu.

The purpose of Mercury’s training is to cover as much of the basic concepts as possible in a short space of time, so they can’t make the exercises too hard; but I don’t like to leave people with the incorrect idea that just because they’ve finished some training they are totally prepared for the level of difficulty they will encounter in the real world.

I have created an exercise where you will need to submit different credit card numbers, but I have made it more difficult as they must be valid credit card numbers.

Start the Credit Card Validation Exercise here.

Some quick notes:

• The exercise is quite simple. If you work intelligently (and do some research), it should not take long.
• Make sure that you really do generate the cards at runtime. Finding 60 different card numbers on the web, trying card numbers at random until you find ones that work, and generating card numbers outside of LoadRunner all miss the point of the scripting exercise.
• Do not run load against my server. Don’t run more than 1 virtual user at a time and be kind with your thinktime and pacing settings.
• Please report functional defects to me via email: stuart at myloadtest dot com
• Once some people have emailed me their solution scripts, I will post some comments about my preferred solution.

Here are some credit card numbers to get you started:

• 30100000000002 (Diners)
• 30100000000010 (Diners)
• 4000000000000002 (VISA)
• 4000000000000010 (VISA)
• 5400000000000005 (MasterCard)
• 5400000000000013 (MasterCard)
• 340000000000009 (American Express)
• 340000000000017 (American Express)

Good luck!

Imagine that you have to create a script for the “Create New User Account” business process. There are several constraints that will make your job difficult…

The username field has the following properties:

• valid characters are a-z, A-Z, 0-9
• usernames are not case sensitive. “Stuart” is the same as “stuart”.
• usernames must start with a letter, not a number.
• usernames may be a maximum of 8 characters long.
• username must not be a duplicate

Some other information about your testing:

• It is unclear when the database in the test environment will be refreshed, but it should happen at least yearly. It will definitely not be refreshed for every test run, and it will not be possible to delete any of the user accounts that you have created.
• The business process takes a minimum of 1 minute to run from start to finish.
• There will be a maximum of 99 virtual users executing the business process in the scenario.
• It is possible that virtual users will execute the business process at exactly the same time.

How do you create usernames that meet all of the above criteria without creating any duplicates? Create a script or provide a descriptive solution.

See the comments section for my solution…

When running a training or mentoring session, people often ask what runtime settings they should use; as if there is a magical list of settings that will always be correct for any testing situation. Obviously you select runtime settings that are appropriate for what you are trying to achieve with your test, but the funny thing is that there are actually a small list of settings that are usually appropriate for most situations. Read on…

Whenever I am using a vuser type that allows multiple actions in a single script, I will create a separate action for each business process and put appropriate percentage weightings on each action. It is very unusual to have to do anything more complicated than this. I don’t usually use the “sequential” option or create blocks unless I need to have fractional percentage weightings for a business process – percentages must be integer values, so to run a business process 0.1% of the time you could create a block that runs 1% of the time, and put an action in the block that runs 10% of the time.

It’s also rare to set a script in a scenario to run for a specified number of iterations (mostly done by time or set to run indefinitely). Generally “number of iterations” is only used when running the script in VuGen.

• “As soon as the previous iteration ends” is used when running in VuGen or when loading/verifying data. Do not use this for load testing
• I have never seen the point of the “After the previous iteration ends” option. Why would you want to run an unknown number of transactions per hour against the system?
• Don’t use the “At fixed intervals”. If something causes your users to become “in step”, they will tend to stay that way and continue to all hit the server at the same time.
• “At random intervals” is definitely the way to go. Obviously for your users to create a certain number of orders per hour the iteration time must average to 3600/num iterations in an hour. Do not make the lower boundary value any bigger than the maximum time it takes to complete the business process, or you will end up creating less transactions per hour than you intend to.

• Logging creates additional overhead on your load generators, and can create huge log files.
• I log absolutely everything when debugging in VuGen.
• When running the script as part of a scenario, I leave extended logging on but change the logging to “Send messages only when an error occurs”. This gives a little more information than turning logging off entirely, and won’t create any additional overhead while everything is running smoothly (and if the system is not running smoothly you are going to need to stop the test and investigate anyway).

• Just like the pacing setting, I think that it is a good idea to put some randomness in your think times.
• I use a random percentage of 50-150% of recorded think times.
• Use “Ignore think time” if you are debugging in VuGen or if you are loading/verifying data.

• This option is ignored by most people. It is used to create a parameter with a given value without having to edit the script (as runtime settings can be overridden in the Controller).
• In the screenshot I have created a parameter of ServerName with the address of the test envioronment. If you were testing in more than one test environment at a time, this would make save some time.

• Continue on error is generally only going to be used if you have written code to do something when you encounter an error. Usually the default behaviour of ending the current iteration and then starting the next one is sufficient). I don’t advise anyone to try to write a script that handles errors in the same way as a real user because it will create a lot of additional work for very little benefit, but doing something simple like writing some useful information to the logs and then calling lr_exit(LR_EXIT_ACTION_AND_CONTINUE , LR_FAIL) can be useful.
• “Fail open transactions on lr_error_message” should always be ticked. If you are raising an error, you should fail the transaction step that you are performing.
• “Generate snapshot on error” is useful. If it is a web script, any error messages should be added to your content check rules.
• Run your virtual user as a thread unless you have code that is not threadsafe or there is some other reason to run your virtual users as a process. The overall memory footprint on your load generators will be higher if you run as a process.
• I never use the “Define each action as a transaction” option. If I want a transaction in my script I will add it myself with lr_start_transaction.
• I never use “Define each step as a transaction” either. If it is a web script, I can use the transaction breakdown graph to get this information, otherwise I will add the transactions myself.

• Not all vuser types have this option available.
• Most of the time my virtual users will use the maximum bandwidth.
• If I want to emulate users with bandwidth constraints, I will do this in a separate scenario.
• Google calculator is handy to calculate bitrates if your bitrate is not available from the drop-down list e.g./ “256 Kbps in bps”

All of the following settings only apply to web-based scripts. Each vuser type will have its own runtime setting options. It is important to know what they mean and how they will influence your test results before running any tests that you plan to report on.

• Some people get confused by the User-Agent (browser to be emulated) setting. If 90% of your users use Internet Explorer 6.0 and the rest use Firefox 1.5, you don’t have to change the runtime settings for your users to match this. All it changes is the string that is sent in the “User-Agent” field of your HTTP requests. This is completely pointless unless your application has been written to serve different content to different browsers based on the User-Agent field.
• TODO

• Generally people won’t be using your web applications through your proxy server, so it shouldn’t be part of your test either.
• If you start getting errors that are due a proxy server rather than the system under test, it will just confuse the people who have to fix the problem.
• A proxy server will also make IP-based load balancing ineffective.
• If it’s an intranet application and everyone will be using the application through the company’s proxy, then the proxy server should be explicity declared to be in scope for your load test. You should make sure that you have an identical proxy server for your test environment, or that you have permission to be generating load on a piece of Production infrastructure.

• TODO

• These settings are default values specified by Mercury, rather than being inherited from the web browser that is installed on your workstation. Generally you will not need to change them, but be aware that they are here.

• Download filters are a quick way of preventing your scripts from downloading content from certain URLs or hosts/domains.
• I generally use this feature when the web application in the test environment contains third-party images used for tracking website usage (e.g. images from Webtrends or Red Sheriff etc).
• I think it is better to specify which hosts your script is allowed connect to, rather than which hosts your script can’t connect to (because it’s easy to miss one accidentally, or the application may change and refer to a new third-party domain).
• Use web_add_auto_filter if you want to specify this in your script rather than your runtime settings.

• I have talked about Content Check rules before; I think that if you aren’t using them already, then you are not getting the most out of the LoadRunner feature-set.

As of LoadRunner 8.1, Feature Pack 3, the following Vuser types are available (note that descriptions are provided by Mercury).

Application Deployment Solutions

• Citrix_ICA
  Represent the Citrix ICA protocol communication between the Citrix client and server as GUI events.

Client/Server

• DB2 CLI
  The IBM Call Level SQL Interface to the DB2 family of databases.
• Domain Name Resolution (DNS)
  A system which translates domain names into internet addresses.
• Informix
  IBM’s Informix Database servers.
• MS SQL Server
  Microsoft’s SQL Server using the Dblib interface.
• ODBC
  Open Database Connectivity – a protocol providing a common interface for accessing databases.
• Oracle (2-Tier)
  Oracle database using a standard client/server architecture.
• Sybase CTlib
  A client/server architecture database called via the Ctlib interface.
• Sybase DBlib
  A client/server architecture database called via the Dblib interface.
• Windows Sockets
  The standard network programming interface for the Windows platform.

Custom

• C Vuser
  A generic virtual user which uses the standard C library.
• Java Vuser
  Java programming language with protocol level support.
• Javascript Vuser
  A scripting language used to develop Internet applications.
• VB Script Vuser
  Visual Basic Scripting Edition language – used for programming documents displayed in Web browsers.
• VB Vuser
  Vuser scripts written in Visual Basic language.
• VBNet Vuser
  Vuser scripts written in Visual Basic language.

Distributed Components

• COM/DCOM
  Microsoft’s Distributed Component Object Model (COM/DCOM) for distributed computing.
• Corba-Java
  Java scripts which use CORBA architecture giving applications the ability to work together over networks.
• Rmi-Java
  Remote Method Invocation – a technology providing remote communications between programs written in the Java programming language.

E-Business

• Action Message Format (AMF)
  A Macromedia proprietry protocol that allows Flash Remoting binary data to be exchanged between a Flash application and an application server over HTTP.
• File Transfer Protocol (FTP)
  File Transfer Protocol – a system which transfers files from one location to another over a network.
• Listing Directory Service (LDAP)
  Record directory based authentication, listings, searches and related transactions.
• Microsoft.NET
  Supports Microsoft ADO.NET.
• Palm
  Palm Handheld computers.
• Web (Click and Script)
  Emulation of the communication between a web browser and the web server at user-action level.
• Web (HTTP/HTML)
  Emulation of communication between a browser and Web server.
• Web Services
  Web Services – a software interface defined and described using XML.
• Web/Winsocket Dual Protocol
  A dual protocol incorporating the Web and Winsocket Protocols.

ERP/CRM

• Oracle NCA
  Oracle 3-tier architecture database consisting of Java client, Web server and database.
• Oracle Web Applications 11i
  In addition to Web HTTP/HTML features, supports the user action level API and JavaScript, with a high level of support for Oracle Web Applications 11i environment.
• PeopleSoft Enterprise
  In addition to Web HTTP/HTML features, supports the user action level API and JavaScript, with a high level of support for PeopleSoft environment specifics..
• PeopleSoft-Tuxedo
  An Enterprise Resource Planning system based on the Tuxedo Transaction Processing Monitor, including automatic correlation.
• SAP – Web
  Similar to Web HTTP/HTML, but equipped to work in conjunction with SAP.
• SAPGUI
  SAPGUI protocol for SAP Front End, versions 6.20 and higher.
• SAPGUI/SAP-Web Dual Protocol
  SAPGUI and SAP-Web dual protocol for SAP Front End, versions 6.20 and higher.
• Siebel – DB2 CLI
  A Customer Relationship Management Application based on a DB2 database.
• Siebel – MSSQL
  A Customer Relationship Management Application based on the ODBC database API.
• Siebel – Oracle
  A Customer Relationship Management Application based on an Oracle database.
• Siebel – Web
  Similar to Web HTTP/HTML, but equipped to work in conjunction with Siebel, a Customer Relationship Management Application.

Enterprise Java Beans

• Enterprise Java Beans (EJB)
  Enterprise Java Beans – an architecture for the development and deployment of Java-server components.
• Rmi-Java
  Remote Method Invocation – a technology providing remote communications between programs written in the Java programming language.

Legacy

• Terminal Emulation (RTE)
  Emulation of users who submit input to, and receive output from, character-based applications.

Mailing Services

• Internet Messaging (IMAP)
  Internet Message Application – a protocol which enables clients to read email from a mail server.
• MS Exchange (MAPI)
  Messaging Application Programming Interface designed to allow applications to send and receive email messages.
• Post Office Protocol (POP3)
  A protocol designed to allow single computers to retrieve email from a mail server.
• Simple Mail Protocol (SMTP)
  Simple Mail Transfer Protocol – a system for distributing mail to a particular machine.

Middleware

• Jacada
  A gateway server providing clients with a Java user interface to mainframe applications.
• Tuxedo 6
  Tuxedo 6.x Transaction Processing Monitors.
• Tuxedo 7
  Tuxedo 7.x Transaction Processing Monitors.

Streaming

• Media Player (MMS)
  Streaming data from a media server using Microsoft’s MMS protocol.
• Real
  A protocol used to transfer streaming data from a media server.

Wireless

• i-mode
  NTT DoCoMo’s technology for accessing the Internet on a mobile phone system.
• Multimedia Messaging Service (MMS)
  Multimedia Messaging Service – used for sending MMS messages between mobile devices.
• VoiceXML
  A standard for voice-based communication between applications.
• WAP
  Wireless Application Protocol – used for Web-based, wireless communication between mobile devices and content providers.

The Virtual Table Server (VTS) is an in-memory database that can be used to share data between virtual users. I have found this useful when I need my VUsers to pass data to each other; it’s certainly a much neater solution than writing to a file and having each VUser check the value in the file on every iteration.

Here is what the documentation has to say about it…

VTS Virtual Table Server (version 2.10)

Virtual Table Server (VTS) first edition, introduced the capability for LoadRunner virtual users, WinRunner and XRunner, to communicate in real time. Data extraction and communication was simplified but limited to column/queue level operations. The new Virtual Table Server II provides a higher degree of data manipulation and 5-10 times better performance. From row level queries, retrievals, updates, insertions, unqueue entries, to database access, VTS II provides the functionality and ease for enhanced inter-process, inter-virtual user communication

One of the limitations of LoadRunner is that a parameter file cannot be shared between different scripts, or between instances of the same script running on different load generators. VTS is an easy way to maintain a single data pool. This is less of a problem if you are using Performance Center, as it is smart enough to allow the same script to run on multiple generators and still maintain the “Select next row: unique” property.

Another limitation that the VTS solves is the ability to maintain information about which data values have been used *between* runs. If you have a script that consumes data (maybe it deletes records), data values can only be used once. This means that every time you run the script, you have to update the data file. Using VTS, you can mark the data as “dirty”, so the next time the script is run, it will use a different value.

Virtual Table Server (version 2) can be downloaded from the Mercury Support website. Note that even though the software is provided by Mercury, it is not officially supported.

Installation is simple, and the server can be configured to run on any port. The API and examples provided with the software work with C-based scripts. A Java API is also provided. JavaScript and VBScript vuser types are not supported.

The API is simple, but unfortunately does not allow you to write SQL queries; instead you must use the functions provided, like lrvtc_retrieve_row() and lrvtc_query_row(). Finally, the VTS can import from and export to a regular database (anything with an ODBC driver), or a text file.

This is a useful tool for anyone who uses LoadRunner to be aware of.

Useless trivia time…

Back in 1983, Broderbund released a simple (and hugely popular) platform game called Lode Runner.

Many years later, Mercury Interactive released a hugely popular load testing tool called LoadRunner.

Notice that the animated figures look very similar. Do you think that this is subtle homage, or just a case of all stick figures looking alike?