Shift left Testing What, Why, and How to Get Started

Modern software development practices integrate testing into the development cycle. Testing can be everything from a unit test on a single piece of code to performance testing the entire software. In this post, we explore why testing has “shifted left” or earlier in the development process and how you can implement load testing into your development rituals.

In the past, nearly all software development followed a linear progression. The so-called Waterfall Method is a project management workflow that splits projects into five steps and focuses on completing each step before starting the next:

1. Requirements (specifications)
2. Design
3. Implementation
4. Testing
5. Maintenance

By the early 2000s, this methodology started to fall out of favor in software development with the advent of the Agile methodology and its derivatives.

Agile software development breaks projects into smaller chunks and iterates toward a complete solution. If you are involved in software development, you have heard of Agile or one of its variants, such as Scrum, Lean, Kanban, and probably others!

Why was there a transformation in software development? As with many things in life, costs.

The Waterfall method consumes resources at each step and presumes no need to revisit the earlier steps. You could go through the entire development cycle and determine afterward that the product doesn’t meet the users’ needs due to some change in requirements or discover a bug that requires completely rethinking the project architecture. Agile seeks to solve this by reassessing the project continuously through at least the first four steps listed above.

You can imagine arriving at step 4: Testing, having invested resources in the first three steps, learning of a complex, fundamental design flaw, and possibly a colossal disaster! Shift left testing aims to prevent this by introducing testing at the earliest stages of a software development project.

The difference between Waterfall and Agile is particularly significant for problems related to application performance testing, including load testing. With the Waterfall method, resolving performance issues introduced during the early stages of development commonly requires a total rewrite of the fundamental application logic for resolution. Often, this is unfeasible or extremely costly and time-intensive.

Why shift left?

Implementing load testing early against the application during development can detect fundamental logic performance issues, even if you conduct relatively small-scale load tests compared to the final expected system usage.

The benefits of shift-left testing are:

• Enables shorter, more Agile test cycles
• Identify defects early in development when they are cheaper to fix
• Reduction in unexpected performance issues towards the end of the project
• Fewer performance issues experienced in production
• Improved quality of the final system

Which tests should shift left?

There are many different types of tests in software development, so how do you choose which tests should be run early in the development process? At a minimum, unit, integration, end-to-end, and load tests. The following is a short description of each type of test:

Unit tests

Static code tests of your application’s smallest chunks (units) to ensure they’re ready to go with no errors. Infrastructure is unnecessary for unit tests.

Recommendation: run unit tests before each commit.

Integration tests

Tests to validate the newly added code (units) work with the existing code when combined (integrated).

Recommendation: Run integration tests with each merge/pull request.

End-to-end tests

End-to-end tests (E2E) test the overall functionality of your application. Usually, E2E tests mimic one user running through the total functionality of your application. These tests cover the application’s user interface, backend services, database, and network communication.

Recommendation: Run E2E tests after every delivery.

Load tests

Load testing is a subset of performance testing. Load testing sends simulated traffic (typically HTTP) to a server and measures its capability to process the traffic. Typical metrics include the server response time and error rate. Gatling Enterprise is a market leader in load testing.

Recommendation: Run load tests before each deployment.

You can and should integrate many more tests, but this is a great core to get you started.

How to shift left your performance testing

Now that we have covered what shift left is and why to shift left, let’s use performance testing as an example of how to shift left. The same general ideas apply to the other types of testing, but since we are performance testing experts, we will focus there.

Code-oriented simulations

Load and performance testing must start as early in the project as possible, ideally from the first build. It is not sufficient for developers to write most of the application in isolation and then hand it over to dedicated testers.

Instead, developers need to be able to write concise performance tests that live in the codebase. Gatling involves developers in the process, enabling developers to write load tests as code quickly.

Integration with CI and CD Tools

To shift left your testing, load test executions must happen as part of your CI build.
Additionally, the load testing solution should be able to pass or fail the build based on predefined non-functional acceptance criteria such as the error rate or response time of transactions.

Gatling features the ability to define these criteria within the load test scripts themselves through assertions. Whenever one or more of these assertions is breached during the build execution, Gatling will automatically update the final build status as appropriate. Thus, the build fails if your application fails to meet a criterion such as response time 95th percentile.

Many open-source and paid CI/CD tools can help you build, deploy, and test your projects. It’s essential to choose one that will work for you and be well-supported over the lifetime of your project. Here are a few of our favorites:

• Jenkins
• TeamCity
• Bamboo
• GitHub Actions
• GitLab CI
• Circle CI

Any of the above are great options, and as a developer or a DevOps team member, you’re probably already using one of these. The best part is that they already have native integrations with Gatling or can be integrated easily using our CI script.

Easily Generate Sufficient Levels of Load

The load-testing tool in your build pipeline must be able to generate a sufficient load to test your application. Gatling’s design efficiently generates load with minimal resources, making it a perfect solution for load testing in your CI pipeline.

When the development of your application is at a more advanced stage, or it is necessary to load test your application in the cloud in a distributed manner, Gatling Enterprise offers the perfect solution to fulfill these requirements and take your load testing to the next level.

Real-Time Monitoring of Load Test Execution

When running load tests as part of your CI pipeline, it is important to be able to access quick feedback on how the load test is progressing. Suppose you observe unexpectedly high response times or large numbers of errors. In that case, you’ll likely want to cancel the build and investigate immediately rather than wait for the load test to finish.

Gatling publishes detailed logs to the console every few seconds. These logs contain high-level information on the test status, including response times of transactions and the number of observed errors. Gatling Enterprise takes this further by offering comprehensive real-time graphical reporting, which multiple users across the business can access.

Testing best practices

Automate small, schedule big

Most of your tests might be short (unit testing, for example), and you should automate these tests to run with every commit. However, especially when it comes to load testing, you will have tests that run for long periods. Rather than automating these tests to run every commit, consider scheduling nightly load tests that run while you’re not working. Gatling combined with a CI/CD tool allows you to schedule your tests during your downtime.

Run parallel tests

You can speed up your CI/CD testing phase by running multiple tests simultaneously. Most of the time, your tests are complementary, and by running them in parallel, you’ll save valuable time and speed up your development cycle.

If you are ready to shift left, we are here to help with the tooling and consulting you need to be successful.