The best solution to shift-left your load-testing

Shift-left testing can be defined as “a method where testing is performed earlier in the development process”. The fundamental reason for testing earlier in the project lifecycle is to discover and fix problems quickly.

By introducing testing early in the software development process, defects can be identified and resolved before they manifest into bigger problems. The longer that a problem remains undetected, the larger it can grow and, ultimately, the more complex and expensive it can be to resolve.

This is especially true of problems related to performance. In the past, when load testing was usually left until the end of the project, the performance issues had likely been prevalent for so long that the only way they could be resolved was from a total rewrite of the fundamental logic of the application. Often this would be completely unfeasible, or extremely costly and time-intensive to implement.

To make matters worse, it will often become apparent that performance issues of this nature could have been identified by implementing early load testing against the application being developed. Even if that load testing was only at a relatively small subset or scale of the final expected system usage, the majority of these critical issues still have the potential to be uncovered early on.

The benefits of shift-left testing can be summarised as:
– Enables shorter, more Agile test cycles
– Identify defects early in development, when they are cheaper to fix
– Reduction in unexpected performance issues towards the end of the project
– Fewer performance issues experienced in production
– Improved quality of the final system

We can see that shifting our testing efforts left is important for all testing types. So what is the best solution to shift-left load testing? To answer this question, let’s identify the characteristics that a load testing tool should possess to facilitate the shift-left.

Code-oriented simulation crafting

Load and performance testing must start as early in the project as possible, ideally from the first build. It is not sufficient for developers to forge most of the application in isolation, and then hand it over to dedicated testers for load testing.

Instead, developers need to be able to write concise performance tests that live in the codebase. Gatling involves developers in the process and enables developers to write load tests as code easily.

Integration with CI and CD Tools

To shift-left your testing, it is imperative that load test executions happen as part of your CI build.
Additionally, the load testing solution should be able to pass or fail the build based on predefined non-functional acceptance criteria such as the error rate or response time of transactions.

Gatling features the ability to define these criteria within the load test scripts themselves through assertions. Whenever one or more of these assertions is breached during the build execution, Gatling will automatically update the final build status as appropriate.

Easily Generate Sufficient Levels of Load

The load testing tool that you are using in your build pipeline must be able to generate the required levels of load to sufficiently load test your application. Gatling is designed to efficiently generate tremendous levels of load and throughput from a single instance, making it a perfect solution for load testing in your CI pipeline.

When the development of your application is at a more advanced stage, or it is necessary to load test your application in the cloud in a distributed manner, Gatling Enterprise offers the perfect solution to fulfill these requirements and take your load testing to the next level.

Real-Time Monitoring of Load Test Execution

When running load tests as part of your CI pipeline, it is important to be able to access quick feedback on how the load test is progressing. If you are observing unexpectedly high response times or large numbers of errors, you’ll likely want to cancel the build and investigate right away, rather than waiting for the load test to finish.

Gatling publishes detailed logs to the console every few seconds. These logs contain high-level information on the test status, including response times of transactions and the number of errors that have been observed. Gatling Enterprise takes this further, by offering comprehensive real-time graphical reporting that can be accessed by multiple users across the team and broader business.

Summary

Shifting your load testing left has tremendous and proven benefits to the software development process, as highlighted at the beginning of this article. Choosing the correct load-testing solution is important to making this transition a success.

Discover how to convert a LoadRunner scenario

Learn more about Gatling, the best developer tool to load test your applications, at gatling.io

The Gatling team