Sylvia is a very good cook. She makes a particularly fine roast ham, using a family recipe that begins, unusually, by cutting a section off both ends of the meat. One day, a friend dropped by while she was preparing the dish. “Why are you doing that?” he asked, as Sylvia carefully trimmed that ham, “Because that’s the way my mother always does it,” said Sylvia. But it got her wondering – why did her mother start that way?
A few days later Sylvia was at her mother’s house. “When you make that special ham, why do you start by chopping the ends off the meat?” she asked. “Because that’s the way my mother taught me,” came the response. Sylvia knew she had to get to the bottom of this. She picked up the phone and called her grandma. “Tell me Gran, when you used to make your roast ham, why did you always start by cutting the ends of?” The old woman paused for a moment in recollection. “Because I didn’t have a big enough pan,” she replied.
I’ve been thinking a lot about Methodologies lately, and the above story from New Scientist magazine (April 1st, 2006) makes the point perfectly that blindly following a process creates cases where you end up doing things that make no sense in a different situation.
Software pundit Joel Spolsky seems to discount the idea of having a Methodology at all – it can be replaced with a quality team of talented people with solid experience. But experience is a dear teacher; giving talented people something that will give them a framework and help them avoid the non-obvious traps has to have some sort of value, right?
Just about every methodology document I have read focuses very heavily on what to do. First you do step A, then step B, if condition C happens, then do step D. This prescriptive format is great if every project you do is exactly the same, but as soon as you encounter something different, you are cast adrift – doing things that make no sense, and groping your way through processes you have had to invent yourself to cover the gaps.
Because most Methodology documents usually seem to be written with one project in mind, they don’t generally bother outlining why the different activities are necessary – it’s probably considered pretty obvious when you only have a single project as an example. But if you explain the why, any halfway bright person can usually figure out the how for themselves. And, it’s harder to fall into the trap of rigidly following a process when it is clear which parts of the process are relevant to your situation.
My current feeling is that the core parts of a performance testing Methodology should fit on a few pages, and be built around some simple key principles like “find problems early” and “demonstrate value” (especially important for a consultancy company). Obviously there will be a need for supporting documents like templates and technical notes, but if it not possible to read through the core process in less than 10 minutes, then something is wrong.
I will let you know how my theories stand up to reality as I do more work in this area, but if anyone out there has any feedback or experiences they would like to share, it would be greatly appreciated.