My Load Test

I’m thinking of writing a WinRunner Coding Standards document (as if there weren’t enough of them out there already), but with more of a focus on why things should be done a certain way rather than just dictating how code should be written.

Here is a sample section. Is anyone interested in reading a document like this?

No magic numbers

Here’s a quick quiz for you. What does the numeric argument to the tl_step function below set the test step status to?

tl_step(“Check that transaction completed successfully.”, 0, “Transaction did not complete successfully.”);

If you answered “fail”, you probably figured it out from reading the other arguments to the function – and you would be wrong. If you answered “pass”, you probably need to spend less time reading the WinRunner manual and more time actually testing. The tl_step function interprets a 0 value as a pass, and any other number as a fail.

Accidentally using a pass value instead of a fail value is quite a nasty bug. If a test script fails when it should not, someone will investigate and find the mistake in the script but if it passes when it should not, the first you will hear of it is when a user discovers a defect in Production. No one reads the comments in the WinRunner Test Results unless something fails.

Here’s another quiz. What does the following line of code do?

while(get_mouse_cursor()==65557);

Do you give up? Obviously WinRunner will continuously loop until the return value of the get_mouse_cursor function is no longer equal to 65557, but what does this magic number 65557 really mean? Whoever has to read your test scripts when you’re gone won’t have to spend time pondering this sort of question if you use a descriptive constant name instead of a unintelligible value.

while(get_mouse_cursor()==HOURGLASS);

A good rule to follow is that no-one who has to read your scripts should have to know what a number means, unless it is obvious from the way it is used (like a time delay, or an index). This applies to the standard WinRunner functions and any that you create yourself. Some of the example documentation even uses magic numbers instead of the WinRunner constants; hopefully this will be fixed in future versions of the tool.

The wr_gen compiled module in the lib directory of your WinRunner installation defines all the constant values used by WinRunner. Some common ones are…

• TRUE, FALSE
• PASS, FAIL
• ON, OFF, TOGGLE
• E_OK, E_GENERAL_ERROR, E_NOT_FOUND etc.
• FO_MODE_READ, FO_MODE_WRITE, FO_MODE_APPEND

Some of these constants are inserted for you automatically while recording or using the function generator, but you must insert many of them manually. Future maintenance and readability should take precedence over the minor inconvenience of replacing a small number of values by hand.

Transcribing an existing language to BNF and creating a parser for it isn’t a trivial task, but it’s not an insurmountable one either. A few years ago I started thinking about what you could do if you developed a parser for Mercury’s Test Script Language (TSL) that drives WinRunner, one of their functional test automation tools.

TSL Code Reformatter
Check test scripts for conformity to project coding standards. Reformat existing test scripts to meet coding standards.

• Indentation, use of whitespace
• Complexity of compound statements
• Check for common errors or misleading code
  if (i = 1) {
  initialiseTest(); // this code should only execute when i is equal to 1, but instead executes every time.
  }
• Use of comments

TSL Code Obfuscator
As TSL is an interpreted (rather than compiled) language, if you give someone your script you have not just given them what they need to run a test, you have given them everything they need to understand and modify your work.

There might be a market for a tool that allows a company to deliver scripts that can be proven to work but do not allow a client to modify the scripts easily. This might be useful in “payment on delivery” situations where the client is not entirely trusted.

The tool could

• Strip all comments
• Remove any unnecessary whitespace
• Change all function names
• Change all variable names

TSL Syntax Highligher
It would be nice to be able to post code-snippets on a webpage with the same font and colouring as they would have in the tool that they were written with. Converting TSL to HTML would probably be the easiest one of these projects as the parser does not have to be perfect, just “good enough”.

It would be nice to have two possible outputs for this tool.

• Old-style HTML (pre-4.01) using <font> tags.
• An XHTML version, where the fonts and colours have been put into a stylesheet. This type of output could also be manipulated and changed to other formats using XSLT.

TSL Code Translator
Convert existing WinRunner scripts into a format usable by another tool. This would be by far the most difficult task in the list. I wouldn’t want to be the person in charge of making this one work.

TSL Documentation Generator
I once worked on a WinRunner project where one of the deliverables was function library documentation in Microsoft Help format (*.chm), which is basically compiled/compressed html.

This was done near the end of the project but, even then, it was a pain to put together. It sucked up at least a man-week worth of time and, because it violated the Pragmatic Programmer’s DRY (Don’t Repeat Yourself) principle which says “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system”, it rapidly went out of date as the function library changed.

The obvious solution would be to have a Javadoc-like tool that allowed you to put specially marked-up comments in the source code that could be automatically extracted and turned into documentation.

But, back in the real world…

Unfortunately, none of these projects are really viable as Mercury licensing prohibits sale of third-party tools that work with their toolset.