More and more IT projects are embracing Continuous Integration. They are finding that the software development practice of building their software after every change, and automatically running unit tests (and maybe GUI-level tests) on every build helps improve software quality and increases their speed of development.
benefits of including performance testing:
LoadRunner with lower maintenance effort. baseline tests
Why do CI
The best tool for the job
In the past, most people who wanted to include load/performance tests in their CI build pipeline used JMeter (an open-source load testing tool), as the cost of purchasing multiple copies of a commercial tool (one for each project) was too high.
Fortunately, HP Software released the free LoadRunner Community Edition in 2014. LoadRunner is an enterprise-grade performance testing tool that is easier to use and more powerful than JMeter. The Community Edition of LoadRunner includes 50 virtual users, which is perfect for the kind of low to medium-load performance tests that should be included in the CI build process.
Integrating with your CI Build Server
HP maintains a Jenkins plugin for LoadRunner which is great, but which lacks some important features. The solution from MyLoadTest can be integrated with all common continuous integration software (Bamboo, Jenkins, TeamCity, etc.) and has features such as trending that are critical when measuring performance over multiple builds.
|Feature||Jenkins Plugin for LoadRunner||Solution from MyLoadTest|
|Run your performance tests using HP LoadRunner scripts||Yes||Yes|
|Pass/Fail the test based on Service Level Agreement (SLA) thresholds||Yes||Yes|
|Define SLAs for response times (average, percentile)||Yes||Yes|
|Define SLAs for server metrics (average CPU utilisation, etc.)||No||Yes|
|Automatically end the test early if it is causing errors or has already failed its SLAs||No||Yes|
|Full LoadRunner Analysis files available for low-level, per-test detail||Yes||Yes|
|Trending of response times and server metrics over multiple tests||No||Yes|
|Track and show changes to scripts and scenarios between tests||No||Yes|
|YSlow! and Google PageSpeed metrics available||No||Yes|
Interested? Contact Us
If you would like to add performance testing to you CI build process, then please contact us. We can get you up and running quickly and easily.