In my first post I gave a very brief overview of the web security testing products offered by HP. Unfortunately people’s understanding of where the products should fit into the software development lifecycle is still weak. This is even the case inside HP.

Here is a current slide from HP Software…

HP's web application security testing products in the software development lifecycle

The obvious, glaring problem with this diagram is that WebInspect is being promoted as a tool to be used in Production. As someone who has spent a long time working in highly technical areas of testing, I have some huge problems with this, and most of my clients will not like this either.

The bottom line is…if you use WebInspect in Production, you are a cowboy and have no place in enterprise-class IT.

  1. If you are waiting until after Go-Live to test something, then YOU ARE DOING IT WRONG! Hopefully you don’t wait to test for functional or performance problems until after you have rolled out your application to your customers. End users will probably not notice if your website has security problems, but if customer data is stolen, then there is no way to get it back. Unlike other defects, you can’t fix theft of customer data by releasing a patch (you can only prevent it happening again).
  2. Defects found in Production are much more expensive to fix than defects found earlier in the software lifecycle. The earlier you find (or prevent) the defect, the less it costs to fix. DevInspect helps prevent security problems during the Build phase through code analysis and by providing secure components, but it does not allow you to conduct a thorough audit of your web application. WebInspect should be firmly targeted at the Test phase (I will discuss QAInspect shortly). Also, if you find problems in Production and the development team fixes them, then it will be necessary to run another round of functional and performance testing to verify that nothing has been broken by the security fixes.
  3. When WebInspect does a crawl and audit of a website, it tests input points by submitting data. Larry Smith likes to tell a story about a company that used WebInspect in Production, and ended up generating 60,000 emails, and previously I mentioned that a Google search for one of the values that WebInspect uses for form submissions returns ~87,500 results.  You do not want to create garbage data in your Production database. This might not be a problem if you have a basic brochure-style website with no points for user input, but if you run an important website with online ordering (the sort of website you really must security test), then garbage data will cause lots of headaches.

The obvious question is “Isn’t QAInspect already targeted at the Test phase? Why does WebInspect need to be used during this phase too?”. Well, I’m glad you asked…

First, let’s have a look at comparative pricing of these products for a single seat perpetual license. QAInspect is around 25% of the price of WebInspect, so obviously WebInspect is offering something that QAInspect is not.

Price comparison of DevInspect, QAInspect, and WebInspect

QAInspect is geared towards users who are not expected to have any understanding of web security testing. It is a fully automated push-button testing tool that is launched from Quality Center. It will scan an application for known vulnerabilities and execute heuristic attacks for some classes of problem (like SQL injection). There are many vulnerability types that are impossible to create automated tests for, and that is where a skilled security tester armed with a copy of WebInspect really shines.

This leads to some conclusions…

  1. DevInspect is good for preventing security problems during the Build phase (code analysis, secure components). Like QAInspect, it can also perform a crawl and audit of the web application for known vulnerabilities and some classes of problem (using heuristics). It is a low-cost product, so each web developer could be given a copy. It would be useful for preventing/detecting web security problems early, but it has no chance of finding some classes of vulnerability.
  2. WebInspect allows a skilled user to perform a thorough security audit of a web application through automated scanning and tool-assisted testing. This ideally happens in a test/pre-Production environment before a new version of the software is released.
  3. QAInspect is simple enough for anyone to use but, as it is a fully automated solution, will not detect every class of security defect. It is cheap enough that it makes sense for a project (that uses Quality Center) to buy a copy so that they can run a regular regression test for security problems.
  4. As the crawl and audit part of DevInspect and QAInspect will miss certain classes of defect, it makes sense to do a pass or two with WebInspect with a skilled security tester driving it. This fact also makes it highly irresponsible to sell either of these products to a company that does not already have WebInspect.
  5. So…use DevInspect to find problems early in the lifecycle. Once the software is close to feature-complete and stable enough for functional testing, then it would be great to do bring in your expensive security testing guru and do a pass with WebInspect. As you don’t want to keep an expensive security tester hanging for too long, you can get your regular testers to run regular overnight regression tests on each build. The security tester will need to be available to investigate any defects found, and eliminate any false positives from the report. Before deploying the application, a final pass should be executed with WebInspect. Any future releases will be regression tested using QAInspect and, depending on what has changed, can be fully regression tested using WebInspect (or WebInspect could be used to do focussed tests on new functionality).

 

Published On: June 5, 2008Tags: ,

5 Comments

  1. Linden June 22, 2008 at 7:37 pm - Reply

    HP should really push WebInspect to be used both in the Test and Production stages of the SDLC. Compliance standards often necessitate testing of in-production systems so there is some merit to what HP is saying.
    To make the “proper” diagram even harder to draw, compliance standards such as PCI DSS, necessitate code revision on a minimum quarterly basis as well as whenever a code change has been made – thus DevInspect can be pitched as useful within the Production stage too!

  2. Gareth Davies July 9, 2008 at 10:45 pm - Reply

    I do not think that diagram should be a fluid one as it is shown.

    I think they are pitching WebInspect as a standalone product to test against current WebApplication that were not developed or tested using the other two products.

    Good read though!

    I am trialing DevInspect at the moment and really like it, however I am finding it hard to get some pricing information.

  3. rada November 5, 2008 at 2:32 am - Reply

    Thorough study and analysis! Is there any comparable difference between QAinspect and the IBM Appscan? If you have analysed it, please let me know.
    Thanks

  4. Larry Smith November 15, 2008 at 12:09 pm - Reply

    I think points about WebInspect are fair but WebInspect was adopted by pen testers and security teams long before the development lifecycle was part of the picture for these tools. QA Inspect is actually identical to WebInspect under the hood in that they share exactly all of the identical capabilities except in the area of certain interactive testing modes like step mode. As a matter of practice I recommend that users of QA Inspect consult with WebInspect users and when required scan with WebInspect. QA Inspect helps to automate the process for a qa tester but complex application scans cannot be peformed blindly and thats the major difference between the tools.. the QA Inspect scan result cannot be viewed except as a set of defects in quality center or to import them into and view them with WebInspect… DevInspect on the other hand increases the code coverage of testing by combining static analysis with the black box testing methods of WebInspect…
    anyway, good comments,, thanks for the thread

  5. Lead Business December 20, 2008 at 8:21 am - Reply

    Well, looking at slide now I get what all three software kits are for. thank you. Actually I find this strange too, to have two programs to test the same things.

Leave A Comment