My Load Test

Sylvia is a very good cook. She makes a particularly fine roast ham, using a family recipe that begins, unusually, by cutting a section off both ends of the meat. One day, a friend dropped by while she was preparing the dish. “Why are you doing that?” he asked, as Sylvia carefully trimmed that ham, “Because that’s the way my mother always does it,” said Sylvia. But it got her wondering – why did her mother start that way?

A few days later Sylvia was at her mother’s house. “When you make that special ham, why do you start by chopping the ends off the meat?” she asked. “Because that’s the way my mother taught me,” came the response. Sylvia knew she had to get to the bottom of this. She picked up the phone and called her grandma. “Tell me Gran, when you used to make your roast ham, why did you always start by cutting the ends of?” The old woman paused for a moment in recollection. “Because I didn’t have a big enough pan,” she replied.

I’ve been thinking a lot about Methodologies lately, and the above story from New Scientist magazine (April 1st, 2006) makes the point perfectly that blindly following a process creates cases where you end up doing things that make no sense in a different situation.

Software pundit Joel Spolsky seems to discount the idea of having a Methodology at all – it can be replaced with a quality team of talented people with solid experience. But experience is a dear teacher; giving talented people something that will give them a framework and help them avoid the non-obvious traps has to have some sort of value, right?

Just about every methodology document I have read focuses very heavily on what to do. First you do step A, then step B, if condition C happens, then do step D. This prescriptive format is great if every project you do is exactly the same, but as soon as you encounter something different, you are cast adrift – doing things that make no sense, and groping your way through processes you have had to invent yourself to cover the gaps.

Because most Methodology documents usually seem to be written with one project in mind, they don’t generally bother outlining why the different activities are necessary – it’s probably considered pretty obvious when you only have a single project as an example. But if you explain the why, any halfway bright person can usually figure out the how for themselves. And, it’s harder to fall into the trap of rigidly following a process when it is clear which parts of the process are relevant to your situation.

My current feeling is that the core parts of a performance testing Methodology should fit on a few pages, and be built around some simple key principles like “find problems early” and “demonstrate value” (especially important for a consultancy company). Obviously there will be a need for supporting documents like templates and technical notes, but if it not possible to read through the core process in less than 10 minutes, then something is wrong.

I will let you know how my theories stand up to reality as I do more work in this area, but if anyone out there has any feedback or experiences they would like to share, it would be greatly appreciated.

A long time ago on a project far far away, I saw something that gave me food for thought. Following an initial baseline performance test, the project was in a tuning cycle, with some very bright performance engineers doing everything they could to drag the application at least somewhere near the performance targets.

The performance requirements they were trying to meet weren’t unreasonable. They were building a replacement system that was almost functionally equivalent to the system it was replacing, and the requirements had been taken from a baseline test of the old system.

As the tuning progressed, the lead developer from one of the system areas had been keeping a spreadsheet tracking the improvements that were made to the response times for each step of the key business processes. I put this down to him being very conscientious, and gave it no further thought…

…until one morning I came into work and some of the biggest performance defects were closed – a comment in each defect said “accepted by the Business”.

Response times were nowhere near the requirements, so I was sure that the lead developer must have used the Force (or was holding someone’s dog to ransom or something).

But it turns out that he just had a much better understanding of human psychology than he did of performance tuning.

In his pop-psycology book Influence: The Psychology of Persuasion, Robert Cialdini explains:

The general rule says that a person who acts in a certain way toward us is entitled to a similar return action…one consequence of the rule is an obligation to repay favours we have received. Another consequence of the rule is an obligation to make a concession to someone who has made a concession to us.

So by using his spreadsheet to show that the performance had improved significantly, the developer put the business owner in a position where he felt obliged to meet the developer “half way” and accept response times that were four times what he had originally asked for (even thought there was still three months to go before Go Live).

I’m sure it was just a trick of the light that made the umbrella handle poking out of his bag look suspiciously like a lightsaber.

I recently heard an anecdote that reminded me of a tip from one of my favourite software-related books – The Pragmatic Programmer.

In it, the authors advise readers to…

Sign Your Work
Craftsmen of an earlier age were proud to sign their work. You should be, too.

Their emphasis is on taking pride in your work and accepting responsibility for it too. But the anecdote I heard showed that signing your work can be useful advertising too.

The consultant telling me the story had just received a call from a company he had worked at many years ago, asking him if he was available to do some more work for them.

All the people he had worked with had long since moved on, but the person assigned to review his old test scripts had found his name in a comment in the source code. Since they were looking for a consultant to do the work anyway, he was suddenly at the top of their list.

From now on, I am adding a small header with

• Full Name
• Company Name
• Email Address
• Mobile Number

to every piece of code I write.