When you set out on a load testing exercise of your web application, there are a number of metrics you will look to measure. These metrics will assist you in uncovering and ultimately resolving potential problems in the application’s performance.
Examples of the metrics that you should measure include:
- Transaction Response Time
How long does each transaction in your user journey take to respond? Examples of a transaction could be logging into the system, adding an item to a shopping cart or completing the checkout process. If a transaction is made up from multiple API calls, these can either be grouped together to get an overall indication of the response time for a particular transaction or analyzed individually.
It is common for the average transaction response time to be measured and reported on when load testing. While this metric does have some merit, you should be aware of its limitations. An average response time can be easily skewed by one or two data outliers, resulting in a false impression that the response time is either far greater (or smaller) than the one being reported on.
A much better metric to use for response time is the nth percentile. Most load testing tools will report transaction response times on a variety of percentiles, such as the 70th, 80th, 90th percentile. This metric reads as “90% of transactions completed in this time or less”, and paints a far more accurate description of the response time and overall performance of the system that will be experienced by users.
- Application Throughput
This measures the number of transactions that can be put through the system over a specified period of time (typically measured in transactions per second). The throughput of a system will give an indication of how much traffic the system can effectively process over any given period.
- Error Rate
When an application is running at different levels of load and throughput, this metric will give you an indication of how many errors are occuring. You can also differentiate by the types of errors that occur at different loads. Examples of error types include timeouts, bad requests or deadlocks as a result of too many processes attempting to access the same resource concurrently.
- Server Resource Utilisation
During the execution of the load test, it is important to instrument and measure the performance of the backend systems that make up the application. This instrumentation is carried out by dedicated system monitoring tools. Examples of metrics that will be measured for server resource utilisation include CPU % used, memory % used, network I/O etc.
We have now identified some of the criteria that we might use to measure the performance of a system through load testing. With that in mind, there are three distinct, common objectives that you may aim to achieve through your load testing efforts:
Suppose that you are about to take out a mainstream advertisement, or you envisage your customer base will expand significantly in the next six to 12 months. Here, the objective of the load test is to assess whether your current system can support this anticipated increase in load whilst still operating at a satisfactory level of performance.
When a system has experienced a major outage, it can be necessary to recreate this situation in a controlled environment. With appropriate monitoring orchestration of the system in place, the data produced from load testing can be used to identify performance bottlenecks. Subsequently, the same load test can then determine the effectiveness of the performance fixes introduced to remedy these issues.
A robust load-testing strategy allows you to train your system’s ability to handle different situations before these occur. A common load-testing objective is to assess the effectiveness of your deployment, monitoring and operational procedures. Adjustments can be made as necessary, with additional load-test scenarios executed to assess their impact.
Learn more about Gatling, the best developer tool to load test your applications, at gatling.io
The Gatling team