Imagine that you are load testing the website for a new loyalty card scheme. Millions of people will be mailed a loyalty card, and they will need to register their details online, along with their card number. Each card number may only be used once. So, every time you run your test, you will consume a large amount of card numbers, and you must be sure to use unregisterd card numbers for each test.

This seems like an obvious time to use the File parameter type, with the parameter properties set to “Select next row: Unique”, and “When out of values: Abort vuser”.

Unique file parameter options

So you request a whole lot of unused card numbers from your DBA, insert them into your File parameter and run your LoadRunner scenario. LoadRunner ensures that all the card numbers are only used once (even though you have hundreds of virtual users), and all of your Register Card business processes run successfully. BUT, when you run your test for the second time, all of your Register Card business processes fail, with the website displaying an error message of “We’re sorry, this card has already been registered”.

You have just discovered that LoadRunner does not make the parameter unique between tests; the parameter will only be unique within the LoadRunner scenario that is currently running. This is a problem anytime you have consumable data. To work around this problem, you really have a few options…

  1. Restore/rollback the database: This is hugely impractical, as it means hours of environment outage when done in a real test environment with Production-sized data volumes. Funnily enough, this solution is seriously proposed in the official HP training for LoadRunner. Maybe the person who wrote the training hadn’t done much real load testing.
  2. Get new data: Each time you wanted to run a test, you could ask your DBA to provide you with new data, or you could run a query against the database and do it yourself (always best to be as independent as possible). Then you could delete the current parameter values, and cut and paste the new values into your parameter file.
  3. Remove the used parameter values from your parameter file: This is the option that we will examine further…

Removing the used data from the parameter file is less straight-forward than you might think.

Diagram of LoadRunner's unique file parameter

When a File parameter is unique, LoadRunner will divide the data evenly into blocks and assign each block to a virtual user when it starts running. Each vuser will use the data in their assigned block sequentially.

This means that the used values aren’t all at the start of the file; they appear in chunks, spread evenly throughout the file (unless your scenario has a single virtual user).


Removing the used values from the parameter file by hand in Notepad is not really practical (unless there is only a few values). I know of three ways to make this task a little easier…

Method 1: Write used values to a text file. Remove used values using Excel

Using the standard library functions for writing to files (fopen, fprintf, fclose), modify your script to save the parameter values it consumes to a file.

Remove used data using Excel

After your test, you can remove the used values from your VuGen script’s parameter file by following these steps in Excel:

  1. Cut and paste all the values from your parameter data file into column A.
  2. Cut and paste your used values into column C.
  3. Enter this formula into cell B1:
    =IF(ISERROR(MATCH(A1,C:C,0)),A1,"")

    The Excel functions used above are:
    MATCH( value, array, match_type ): value is the value to search for in the array. array is a range of cells that contains the value that you are searching for. match_type is optional. It is the type of match that the function will perform. Using 0 for this argument means that the MATCH function will find the first value that is equal to value. The array can be sorted in any order.
    ISERROR( value ): value is the value that you want to test. If value is an error value (#N/A, #VALUE!, #REF!, #DIV/0!, #NUM!, #NAME? or #NULL), this function will return TRUE. Otherwise, it will return FALSE.
    IF( condition, value_if_true, value_if_false ): condition is the value that you want to test. value_if_true is the value that is returned if condition evaluates to TRUE. value_if_false is the value that is return if condition evaluates to FALSE.

  4. Fill all the cells in column B with your formula. Click in B1, and move the mouse cursor to the bottom right corner of the cell until the white cross turns into a black cross. This is called the fill handle. Click and drag the fill handle down the column to fill it with your formula. Alternatively, double click on the fill handle and it will fill all cells in the column until it runs out of rows in the column to its left (this is the reason for the order of the columns that I have specified).
  5. Remove the blank rows with a filter. Select column B and click on Data > Filter. Change the filter to exclude blanks.
  6. Cut and paste all the values from column B back into your parameter data file.

Note: If you are going to write used data to a text file, it is important to write the value to the file at the start of your iteration rather than at the end, otherwise you may leave the data in an unusable state if your vuser stops half way through the business process.

Method 2: Write used values to a text file. Remove used values from parameter file with command-line tools

After using the Excel method for a while, I realised that I could probably do the same thing with a 1-line shell script, using pipes and some combination of sort, uniq, diff, and awk; instead I found that *nix has a neat tool called comm, which finds lines that are common to two files. Perfect.

Note that this solution requires you to find Unix command-line utilities that have been ported to Windows. Here is the solution in a batch file…

REM This batch file will remove used {CardNumber}s from the cards.dat parameter file.
REM Note that the parameter file will be overwritten each time this batch file is run.
REM Be sure to update the script path if you change the name of the script!
SET used_file="C:\LoadRunner\data\used_cards.txt"
SET param_file="C:\LoadRunner\scripts\register_loyalty_card_20120512_01\cards.dat"
REM comm arguments:
REM -1 : suppress lines unique to FILE1
REM -2 : suppress lines unique to FILE2
REM -3 : suppress lines that appear in both files
comm -3 %param_file% %used_file% > %param_file%

If you are really clever, you could make this batch file run automatically by putting it inside a system() call in a VuGen script that runs at the end of your scenario (i.e. after all the other groups have finished).

Sidenote: If you job involves processing text (data, logs, etc) and you aren’t familiar with the various Unix tools that you can glue together to manipulate data, then you are probably making life unnecessarily hard for yourself.

Method 3: Use a database to store values. Mark data as “used” at runtime

One of the frustrating things about the solutions which involve removing values from parameter files is when you run your test for 2 minutes and then realise that you forgot to change a critical runtime setting back to its correct value the last time your ran your script in VuGen. Now you need to stop your test and clean out your used values before you run it again.

You can avoid this problem entirely by removing the values as they are used (i.e. at runtime), with no manual steps. To do this, you must abandon LoadRunner File parameters, and use some kind of data store, which you will access using function calls in your script (note that some other performance testing tools handle this problem automatically with a “Data Pool” feature).

I prefer to use MySQL, and have written various functions to add and remove data. Using MySQL involves quite a bit of work upfront to set up your database and the tables to store your parameters, but this effort is repaid if you have to run more than a few tests. The official (but unsupported) solution from HP/Mercury is their Virtual Table Server (VTS2), which is easier to set up than MySQL, but is clunky to solve this “consumable data” problem with, and has performance problems under high load. Note that both of these solutions are for slightly more advanced LoadRunner users.

 

Published On: May 12, 2012Tags: ,

10 Comments

  1. cat /dev/random May 12, 2012 at 2:16 pm - Reply

    Yes to your sidenote! I totally agree!!! If you work with computers, not knowing Unix is like being slightly retarded. You will constantly be doing things the dumb way. If you only know Windows, you should pick up a book and get some education!

  2. VuGen Script Monkey May 12, 2012 at 9:19 pm - Reply

    How do I use a batch file to remove used data if the parameter file has multiple columns?

    • Stuart Moncrieff May 13, 2012 at 1:21 pm - Reply

      I assume that you could do this with use pipes and the “cut” command…and maybe some awk.

      REM Please leave a comment with your code if you figure it out
      

      Unix command line tools for Windows are available here: http://gnuwin32.sourceforge.net/packages/coreutils.htm (GnuWin). Note that there is no need to install Cygwin to use these tools.

  3. DBA stands for "Does Bugger-All" May 13, 2012 at 2:19 pm - Reply

    Is there are reason you can’t just query the application’s database from your script (at runtime) to get unused data, instead of setting up a separate database to store the parameter values?

  4. Anu June 30, 2012 at 1:13 am - Reply

    Hi,
    Thanks a bunch for this information. Really aprreciate the way you explain things.

  5. Divya March 15, 2013 at 3:47 am - Reply

    Hi..

    I was trying to save the data in a file that is present in a shared location using loadrunner script. But i could nt do it. Whereas manually i could create a file in that particular location and use it as a parameter file for my script. But could nt save any resulted values in the shared location.

    And also i could do the same operation i.e to saving values in a file in my local machine thorugh the script . Can you please suggest me a way to do it or is there any issue with that.

    ..

  6. sankar July 22, 2013 at 7:40 pm - Reply

    Hi Stuart, I am facing the below problem in performance center while running for more than 30 users. In log file it has logged the below error for most of users..

    Action.c(469): Registered web_reg_find successful for “Text=Import Progress” (count=3) [MsgId: MMSG-26364]
    Action.c(469): Notify: Saving Parameter “TextCheck = 3”. [MsgId: MMSG-13798]
    Action.c(469): web_submit_form(“ImportProgressAction.do”) was successful, 40106 body bytes, 311 header bytes, 96 chunking overhead bytes [MsgId: MMSG-26385]
    Action.c(476): Notify: Parameter Substitution: parameter “TextCheck” = “3” [MsgId: MMSG-13992]
    Action.c(477): Notify: Transaction “Import_ImportProgress” ended with “Pass” status (Duration: 5,1066 Think Time: 5,0032 Wasted Time: 0,0001). [MsgId: MMSG-16870]
    Ending action Action. [MsgId: MMSG-15918]
    Clearing extension two_way_comm.dll by library request. [MsgId: MMSG-10497]
    Clearing extension two_way_comm.dll by library request. [MsgId: MMSG-10497]
    Clearing extension launcher.dll by library request. [MsgId: MMSG-10497]
    Clearing extension launcher.dll by library request. [MsgId: MMSG-10497]
    Clearing extension lrwreplaymain.dll by library request. [MsgId: MMSG-10497]
    Clearing extension lrwreplaymain.dll by library request. [MsgId: MMSG-10497]
    Clearing extension run_time_context.dll by library request. [MsgId: MMSG-10497]
    Clearing extension run_time_context.dll by library request. [MsgId: MMSG-10497]
    Clearing extension cciext.dll by library request. [MsgId: MMSG-10497]
    Clearing extension cciext.dll by library request. [MsgId: MMSG-10497]
    Clearing extension parameng.dll by library request. [MsgId: MMSG-10497]
    Clearing extension parameng.dll by library request. [MsgId: MMSG-10497]
    Clearing extension faserver.dll by library request. [MsgId: MMSG-10497]
    Clearing extension faserver.dll by library request. [MsgId: MMSG-10497]
    Clearing extension vusr_log.dll by library request. [MsgId: MMSG-10497]
    Clearing extension vusr_log.dll by library request. [MsgId: MMSG-10497]
    Clearing extension LrXml.dll by library request. [MsgId: MMSG-10497]
    Clearing extension LrXml.dll by library request. [MsgId: MMSG-10497]
    Clearing extension lrun50.dll by library request. [MsgId: MMSG-10497]
    Clearing extension lrun50.dll by library request. [MsgId: MMSG-10497]
    Clearing extension trans.dll by library request. [MsgId: MMSG-10497]
    Clearing extension trans.dll by library request. [MsgId: MMSG-10497]
    Clearing extension runlogic.dll by library request. [MsgId: MMSG-10497]
    Clearing extension runlogic.dll by library request. [MsgId: MMSG-10497]
    Clearing extension zext.dll by library request. [MsgId: MMSG-10497]
    Clearing extension zext.dll by library request. [MsgId: MMSG-10497]
    Action was aborted. [MsgId: MMSG-10694]
    Ending Vuser… [MsgId: MMSG-15966]
    Starting action vuser_end. [MsgId: MMSG-15919]
    Ending action vuser_end. [MsgId: MMSG-15918]
    Vuser Terminated. [MsgId: MMSG-15963]
    Service client “254” was unset. [MsgId: MMSG-29444]
    Service client “253” was unset. [MsgId: MMSG-29444]
    Service client “252” was unset. [MsgId: MMSG-29444]
    Service client “251” was unset. [MsgId: MMSG-29444]
    Service provider with id=250 was unset. [MsgId: MMSG-29447]
    Service provider with id=255 was unset. [MsgId: MMSG-29447]
    19/07/2013 23:16:26 (-103997): RTC Context Manager Debug: Unregistered key: ParamDynamicList. [MsgId: MMSG-103997]
    19/07/2013 23:16:26 (-103997): RTC Context Manager Debug: Unregistered key: ParamList. [MsgId: MMSG-103997]
    19/07/2013 23:16:26 (-103997): RTC Context Manager Debug: Unregistered key: Iteration. [MsgId: MMSG-103997]
    19/07/2013 23:16:26 (-103997): RTC Context Manager Debug: Unregistered key: Action. [MsgId: MMSG-103997]

  7. Anton August 14, 2013 at 6:42 pm - Reply

    Hi guys.

    I need to re-use a returnded xml value from one script in 1 or 2 different following scripts.
    Is there a standard way or feature of doing that in LoadRunner? … or will I have to resort to storing to and retreiving from an external location such as a file or DB?

  8. Sandhya March 3, 2015 at 11:24 pm - Reply

    Hi guys,

    I have started using VTS2 with SAP GUI and Web scripts of LR 11.52. I am able Insert the captured values to VTS through web script but with SAP GUI it is not working.

    Could you please suggest any other option or solution to move further.

    I have create data using SAP GUI script and have to use same data in web script in the same scenario. So have tried with VTS2. Unfortunately VTS is not working with SAP GUI scripts.

    Thanks in advance.

Leave A Comment