Understanding workload models for load tests

When it comes to load testing, choosing the right workload model is crucial for accurately simulating real-world user behavior and understanding how your application will perform under different conditions. In this post, we’ll explore the two primary workload models used in load testing:

• open workload model
• closed workload model

We’ll dive into the differences between them, the scenarios in which each is most appropriate, and how they can impact the results of your performance tests.

What Are Workload Models?

Workload models are a strategy that defines how users interact with your application during a load test. They help simulate the conditions your application will face in production, allowing you to identify potential bottlenecks and optimize performance.
The choice of workload model can significantly influence the behavior of your tests and the insights you gain from them.

There are two main types of workload models in load testing:

• Open Workload Model: new virtual users arrive regardless of the existing number of concurrent virtual users.
  Example: e-commerce and news sites
  
  
• Closed Workload Model: new virtual users are queued if the system is at a pre-defined capacity. New users only enter the system when existing users exit.
  Example: ticketing platforms and call centers.

Each model represents a different approach to simulating user activity, and understanding their distinctions is key to selecting the right model for your testing needs.

The Open Workload Model

An open system is a system that has no control over the number of concurrent users. Users keep arriving, regardless of the number of concurrent users inside the system. At some point, if the system starts slowing down, users who are already in the system will take more time to complete their journey, while new users will keep on arriving. Users will then pile up, maybe exponentially, causing your system to eventually crash.

This model is like real-life scenarios where new users can start interacting with the application at any time, independent of the actions or presence of other users. For example, most public-facing websites behave this way.

Characteristics of the Open Workload Model:

Dynamic User Arrival: Users enter the system independent of the system conditions, meaning the system constantly receives new requests at varying rates.

Simulates Public Traffic: Ideal for applications exposed to the general public, such as e-commerce sites or news platforms, where user activity can vary widely.

Focus on Request Rate: In open models, the focus is on the rate of incoming requests rather than the number of concurrent users.

The Closed Workload Model

In contrast, the closed workload model operates with a fixed maximum number of virtual users in the system. Each virtual user performs a series of tasks (or a single task) and then exits the system. Once the system capacity is reached, additional virtual users are queued and only enter as other virtual users exit (1 in-1 out).

This model keeps the maximum number of concurrent users constant, making it easier to simulate a controlled environment where user actions are predictable and manageable.

Characteristics of the Closed Workload Model:

Fixed User Base: The maximum number of concurrent users is predetermined and remains constant throughout the test.

Simulates Controlled Environments: Suitable for applications where user numbers are controlled and relatively stable, such as ticketing platforms.

Focus on Concurrency: In closed models, the focus is on the number of simultaneous users and how the application manages ongoing user sessions.

Picking the correct model

Your system architecture determines which model you should choose. You should select the model that matches the behavior and the architecture of your system under test in production, even if it means reconsidering how you’ve been doing things to date.

Let’s illustrate this by considering the consequences of selecting the wrong model:

Imagine you have an open system and test it with a closed workload model. If the system under test starts slowing down due to a performance issue, the load injector will reduce the arrival rate to keep the number of concurrent users from increasing. In this case, the load injector and system under test are working together to find a balance, and you have a seemingly successful test.

This balance does not exist in the real world. Your users are not load injectors. In production you do not know how your system will respond to an unbounded increase in users. Thus, the results of your test are meaningless.

If you’re using a closed workload model to test a system that’s actually open, you’re introducing an artificial bias. Don’t do this.

Using workload models with Gatling

First an important note, you cannot mix workload models in Gatling. The preceding discussion hopefully makes it clear that the purpose and conditions for each test are drastically different and thus it makes no sense to mix the models for any meaningful load testing strategy.

Using the Closed Workload Model

The closed workload injection profile is contained in the setup block in a Gatling script. The following example injects 3 initial users and maintains 3 concurrent users for the test duration.

   setUp(
            scn.injectClosed(
                  constantConcurrentUsers(3).during(60)).protocols(httpProtocol)
  );

In the following graphs you can see a profile of the user arrival rate and the concurrent users for the preceding code example. Notice how the arrival rate is not constant but the concurrent users is constant during the test!

Using the open workload model

The open workload injection profile is contained in the setup block in a Gatling script. The following example injects and average of 3 users per second for 60 seconds. The number of users at each injection interval is randomized.

setUp(
        scn.injectOpen(
                constantUsersPerSec(3).during(60).randomized())
                .protocols(httpProtocol)
 );

In the following graphs, you can see a hypothetical profile of the user arrival rate and the concurrent users for the preceding code example. Notice how the arrival rate averages 3 users per second but the concurrent users is different and not responsive to the injection rate.

Conclusion

Hopefully this article gives you a better understanding of injection models in load testing. However, the examples are just the beginning. You can achieve much more detailed and complex user scenarios with Gatling’s test-as-code approach.

To keep learning, follow one of our introductory tutorials and then experiment with changing the workload model:   

• Create a simulation with Java
• Create a simulation with JavaScript