Black Friday is approaching fast. With the profound increase in e-commerce transactions this year caused by the COVID-19 pandemic, record-breaking numbers of online transactions are expected in 2020. Is your web application or e-commerce solution ready to handle this make-or-break period in the calendar? If it is, you can expect to gain huge amounts of increased revenue and customers from the Black Friday extravaganza…
But if it isn’t, then it’s your competitors that will be inheriting the extra revenue – and perhaps your once-loyal customers as they permanently switch to a more reliable option….
Practically all website stakeholders can appreciate the value and importance of having high-quality and meaningful load testing. But with an annual event like Black Friday, it’s impossible to understate the importance of keeping your application online and responsive.
A responsive and fast-loading online store inevitably translates to higher revenue. In 2018, France saw online sales increase by +265% on Black Friday, with traffic more than doubling compared to the typical average period.
Not only can you expect more visitors to your site, but users are actually prepared to spend more money. A 2019 study published by Statista shows that 38% of people planned to spend 101-300€ on Black Friday.
To empathise the importance of load testing times when it comes to Black Friday, a study from BigCommerce found than even a one-second delay can cause conversion rates to dip by as much as 7%, as end-users are not prepared to wait when they can simply visit a competitor.
If you are about to enter the Black Friday period with insufficient or even zero load testing in place, then read on as this article is for you.
In this article, we will share 7 load testing tips that will help to ensure Black Friday is a success for your system and your business. Additionally, you will see how to quickly set up and configure a robust, effective and powerful load testing framework from scratch with Gatling.
Let’s get to it!
Tip 1 – Start with an Open-Source Load Testing Tool
Don’t be overly ambitious with your load testing efforts, especially when you are first starting out or are racing against time to prepare for an event like Black Friday. There is often no need to build a testing cathedral or invest massive amounts of money in a commercial-level load testing solution. A few realistic, accurate and comprehensive load testing scenarios created with an open-source tool like Gatling represents a great starting point.
Let’s see how to get up and running in Gatling in minimal time:
Gatling Stand-alone Download
An easy way to get started with Gatling is by using the Gatling Open-Source bundle. Simply download the zip file from the link, and after unzipping open a command prompt or terminal.
Whilst inside the Gatling folder that you unzipped, simply type:
Or on Windows:
This will load Gatling, and scan for any simulations within the folder.
Gatling ships with a few sample simulations that you will see shown to you. You will then be prompted on-screen for a simulation to run:
Simply choose a simulation and Gatling will execute the load testing script.
Gatling Test Results Report
Upon completion of each Gatling test execution, you will notice a link printed to the console for the full test report that has automatically been rendered:
We can open this index.html file to view the rich graphical test report:
Whilst running Gatling with the stand-alone version is great for an initial evaluation, you will quickly want to migrate to using a build tool. Gatling supports Maven, Gradle and SBT
To get up and running in your chosen build tool as quickly as possible, the easiest way is to clone the official Gatling demo repository for your chosen tool, and then start adding your own code to that:
Now that we have Gatling installed on our system, let’s move on to our next tip:
Tip 2 – Start with a Simple Scenario
When starting out, it’s important to keep things as simple as possible. Let’s look at how to write the most simple of testing scenarios in Gatling, where we simply load the homepage of our site.
Note that we will be writing some Scala code in this article, or more specifically Gatling code. If the thought of writing code is uncomfortable to you, rest assured that there is no need to feel intimidated when writing Gatling code.
Gatling ships with a powerful DSL (Domain Specific Language) that makes it simple to construct a detailed load test by writing simple, meaningful and verbose words and phrases.
Let’s look at one of the most simple Gatling scripts you might start out with to prepare your web application for Black Friday:
That is our entire load testing script in these few lines, but let’s break it down line-by-line. We start by importing a couple of packages, which contain all the language definitions needed to make Gatling work:
Next, we define our Black Friday load test simulation class (called ) and we extend this from the special Gatling class:
Inside the simulation class we just created, we now define the protocol we want to use (HTTP in this case) and supply the default parameters. For our script, we are simply supplying the base URL of our web application:
Now we can define the steps of the scenario itself. These are the steps that the virtual user will take on our website when we execute the load test. Because we are aiming for the simplest test possible to help us prepare for Black Friday, we are simply loading the home page here:
Although our scenario above only loads the homepage of our web application, for your Black Friday load test you will likely want to modify it to reflect the expected behaviour of your end-users.
For example, as users will likely be searching regularly for the best deals, you can expect to see a significant increase in the number of searches being performed in your application. You should therefore modify your scenario appropriately so that users perform more searches than normal.
Before we continue, let’s explain the differences between a scenario and a simulation in Gatling terminology:
A scenario describes the steps a virtual user takes through the system i.e. load the home page, click on the product, add to cart etc.
While the simulation is the definition of the load profile that will be executed for the test, i.e. the number of users, the throughput and arrival rate of those users, the test duration etc.
With this terminology in mind, we can look at the block of our script, where we define the load simulation:
The above load simulation will simply run our scenario with a single user, which is a useful default to have in place when creating and debugging your Gatling script. When you are ready to run an actual simulation based on your expected traffic for Black Friday, you will update this block appropriately as seen later on in this article.
With this basic script in place, let’s move on to our next tip:
We need to take time to model this behaviour in the scenario definition of our Gatling script. How this scenario looks will naturally depend on your application, but we can use the built-in Gatling Recorder to record the steps we expect our users to take through our application during Black Friday.
The Gatling recorder acts as a proxy, capturing the traffic that is generated as the user performs various actions on the website, such as searching for a product or going through the checkout process. This generated code is then turned into a Gatling script, which we can use for our scenario.
Now that we have a Gatling script in place with accurate scenarios, we need to think about the next point:
Tip 4 – Execute Different Types of Load Test
Think about the different scenarios and situations your system might likely find itself in during this busy Black Friday period.
Going back to our original simulation that we created earlier, we defined the following:
This load test profile is suitable for when we are creating and debugging our Gatling script. But if we want to make sure our web application is ready for a busy Black Friday, we need to define a suitable injection profile!
Remember that the injection profile is simply a chain of rules, which will be executed in the provided order, to describe the rate at which virtual users execute their test scenario (or “user journey”) through the system.
We need to ask ourselves how we expect users to arrive at our system during the Black Friday period. Do we expect users to arrive at specific time periods throughout the day, or is it more likely to be a steady stream of traffic?
If one of our competitor’s sites were to go offline during the Black Friday period, we could reasonably expect a sudden surge in traffic. Will our system be able to cope with that?
Are we launching a special offer during this promotional period, such as a Black Friday Countdown? When word spreads on social media or otherwise, will this lead to a steadily increasing amount of traffic and load through the system?
Considering these questions and situations will help us to define our injection profile, also known as a type of test. Let’s look at three of the most common ones here:
A stress test, also referred to as a “spike test”, represents a sudden surge in demand on the system. During this period, we can expect many new users to arrive at the system in a short amount of time. In Gatling, we could define this injection profile as follows:
From a Black Friday perspective, we might expect one of these surges to occur if one of our competitors suddenly went offline. Alternatively, a viral social media post or some other form of advertising that alerted customers to a potential offer could see this scenario occur.
To achieve this level of load and throughput in our test environment would likely require multiple load injector machines and support for distributed load-testing. Gatling FrontLine, the Enterprise version of Gatling, would enable us to execute such a test with just a few clicks whilst still using the script we created in the open-source version of the tool.
Soak testing allows us to ensure that our system performance remains stable and consistent for longer periods of increased load. The goal of this type of test is to identify problems in the system like memory leaks or high CPU utilisation, that only occur over a longer period.
A simple Gatling injection profile might look like this:
If we wanted to add a period of ramping-load onto the system before staying at a steady rate, we could combine two Gatling DSL methods:
The idea with Capacity testing is to keep increasing the load on the system until we find the “breaking point”, where transaction response times greatly increase and errors begin to occur.
Having identified this breaking point in our system, we can be confident of handling the expected levels of traffic that Black Friday is likely to bring through our system.
To write a capacity test in Gatling, we can run at a constant throughput for an arbitrary amount of time, then ramp the load to an increased level before continuing with another constant load.
Our injection profile might look something like this:
Instead of having to write out cumbersome code to repeat this process many times, we can take advantage of the Gatling DSL to write an expressive test for us:
Graphically, this scenario would look like this:
Tip 5 – Measure Response Times as Percentiles
Beyond looking at errors and the error rate we see in our load test, the key metric to monitor is transaction response times. These can be monitored broadly in two different ways:
Response time as an average – typically measured as the mean response time
Or as a percentile
Although using the average to measure response times can be tempting, it is an inferior approach, as it can hide the true response time of your system for the majority of your users.
Measuring response times in percentiles is far superior when it comes to load and performance testing. These enable you to give an accurate reflection of what performance is like for the given percentage of your users. In our load testing results, if the 99nth percentile was shown as 100ms, this means that 99% of users experience a response time of 100ms or less.
Presenting metrics to stakeholders in this way is of far greater value. Remember that we want to make sure that the vast majority of users to our site have a great experience, with fast load times and good performance.
Tip 6 – Execute a Distributed Load Test
Assuming you have an application that is internet-facing, your traffic can and will come from many sources across the globe. Although starting out your load testing from a single load injector is a good start, and can easily be achieved with Gatling Open-Source, it is not sufficient to simulate a true real-world load test.
In order to generate the traffic for such a realistic load test, the traffic must come from multiple sources by using a distributed load testing infrastructure. This can typically be achieved by having multiple cloud-based load injectors running across a broad range of geographical availability zones.
Executing a load test of this nature might sound both expensive and cumbersome, but that isn’t actually the case. Using FrontLine, Gatling’s Enterprise tool, you can take your existing Gatling test scripts that were created with the open-source version and run a distributed load test across a range of availability zones in the cloud with just a few clicks.
Gatling FrontLine is available on demand. This means your load testing infrastructure is only deployed and active for the period of your load test executions, significantly reducing the cost of load testing when compared to other solutions.
Black Friday is a perfect reason to start investing in your load testing efforts, but once the event is over you shouldn’t stop there. Surges in traffic or unexpected load on your system could happen any time of the year, and your system needs to be ready to handle this.
Developing load testing scripts with Gatling open-source is an excellent place to start, as we have seen in this article. Once you have those scripts in place and are ready to take your load testing to the next level with distributed load testing on demand coupled with detailed real-time reporting, Gatling FrontLine is the natural solution to progress to.
A minimal investment in high-quality load testing with Gatling FrontLine is highly worthwhile, as although you can never know when that next spike in traffic is likely to come, you want to make sure your system and your business are ready to handle it and take advantage.
If you want to ensure that your web application will be ready for Black Friday, or any other peak traffic period, then high-quality and reliable load testing is essential. The open-source version of Gatling represents a fantastic place to start.