IoT performance testing: Navigating the connected device challenge

IoT deployments have grown past an estimated 30 billion connected devices, roughly three times the number of traditional non-IoT devices. Industry estimates put the share of those devices experiencing performance-related issues at around 64%. From self-driving cars making split-second safety decisions to healthcare devices monitoring vital signs, a broker that falls over during a reconnection storm isn't an inconvenience, it's the difference between a system users trust and one they don't.

Most guides to this topic stay at the level of "IoT is hard because devices are constrained and networks are unreliable," which is true and also not load-testable advice. This one is scoped specifically to load testing: the protocol mechanics, the thresholds worth asserting on, a worked MQTT simulation with real Gatling code, and an honest account of where a tool like Gatling can and can't take you

Key takeaways

• MQTT QoS levels aren't free. QoS 0 is one packet. QoS 2 is a four-message handshake (PUBLISH, PUBREC, PUBREL, PUBCOMP) per logical message. Load test at the QoS level your fleet uses in production, not QoS 0 because it's convenient.
• A device fleet is usually a closed workload model, not an open one. Unlike public web traffic, a fleet has a fixed, known population of devices. Model it as a fixed number of persistent connections, not an arrival rate, and reconnection storms will show up as the demand spikes they really are.
• Gatling doesn't have a native CoAP module. If your fleet speaks CoAP, plan to test at the gateway boundary where it gets translated to MQTT or HTTP, not the CoAP leg itself. Details below.
• AMQP is testable today, through Gatling's official community plugin (also covers RabbitMQ). MQTT is native, with a usage cap on Community Edition (5 users, 5-minute runs) and no cap on Enterprise.
• Retained messages, Last Will and Testament, and clean sessions are all things a normal API load test never has to think about, and all three are places where broker behavior degrades under load in ways a "does it return 200" check won't catch.
• Set thresholds on trends, not snapshots. A broker's queue depth or memory usage at one point in time tells you less than whether it's climbing during a sustained run.

Why smart devices break the rules of traditional testing

Connected products introduce constraints a typical web or mobile load test never has to account for.

Protocol complexity. IoT ecosystems mix MQTT, CoAP, Bluetooth Low Energy, LoRaWAN, and raw TCP/UDP, each with its own payload structure and failure behavior. A test plan built for one protocol tells you nothing about the others.

Resource constraints. Devices run on limited battery, memory, and CPU, which produces duty-cycle behavior: bursts of activity followed by sleep states, rather than the steady request pattern a web load test assumes. A simulation that ignores duty cycling will get the concurrency math wrong.

Bursty, correlated traffic. This is the one that breaks brokers in production. When a firmware update, a regional outage, or a scheduled sleep-wake cycle brings thousands of devices back online within the same window, you get a connection storm, not a gradual ramp. MQTT's own QoS 2 delivery guarantee makes this worse under pressure: each QoS 2 message is a four-packet exchange, so a storm of QoS 2 reconnects and re-publishes generates roughly four times the broker-side message traffic that the device count alone would suggest.

Network variability. Devices operate over cellular, satellite, or unlicensed spectrum with variable latency, jitter, and packet loss, which shows up directly in SLA-critical numbers like command round-trip time and OTA update duration.

When devices fail, users lose trust

The performance gap shows up as a trust gap. Concerns like these keep surfacing in IoT user research:

• Safety concerns about self-driving cars, largely tied to software glitches affecting split-second decisions
• Low confidence in IoT-connected medication delivery in healthcare settings
• Distrust of vital-sign monitoring devices, citing accuracy variation between readings and reality
• Worry about smart-home devices malfunctioning in a way that locks users out of control entirely

None of this is really about whether the device works in a demo. It's about whether it keeps working at 3am during a regional network blip when 50,000 other devices are trying to reconnect at the same time, which is exactly the scenario a load test is supposed to surface before a user finds it.

What to measure

Effective IoT load testing means watching metrics across every layer of the architecture, not just the API surface.

• Device and edge: CPU, memory, and battery draw under sustained load; local queue depth and edge processing latency; device reconnection rate and connection stability.‍
• Network: p95 latency under realistic network conditions, not a clean lab connection; packet loss and jitter; time to reconnect after an interruption.‍
• Cloud and broker: messages per second at peak, not average; messages in flight and throttling percentage; rule-execution and backend processing latency.‍
• End to end: command-to-actuation latency for anything safety- or time-critical; OTA update success rate and duration; data freshness and synchronization accuracy.

Five ways to test IoT systems

There are five real approaches, each with a different tradeoff between fidelity and scale. (If you've seen this framed as "four proven approaches" elsewhere, including in an earlier version of this article, that was a counting error. There are five.)

Matching the approach to your deployment

Not every fleet needs the same protocol, QoS level, or test focus. A rough decision framework:

MQTT load testing with Gatling: a worked example

MQTT is the protocol most IoT fleets run on in practice, and it's the one Gatling supports natively (versions 3.1, 3.1.1, and 5). One licensing note worth stating plainly: MQTT support ships under the Gatling Enterprise Component License, so it runs on Community Edition too, but capped at 5 concurrent users and 5-minute test durations. A realistic fleet simulation, like the one below, needs Enterprise.

Here's a scenario matching a common real-world shape: 10,000 factory sensors, each publishing a reading every 5 seconds, which works out to roughly 2,000 messages per second in steady state.

public class FactorySensorFleetSimulation extends Simulation {

  MqttProtocolBuilder mqttProtocol = mqtt
      .broker("tcp://broker.example.com:1883")
      .clientIdPrefix("sensor")
      .cleanSession(false)
      .keepAlive(30);

  ScenarioBuilder telemetry = scenario("Sensor telemetry")
      .exec(mqtt("Connect").connect())
      .during(Duration.ofMinutes(30)).on(
          exec(
              mqtt("Publish reading")
                  .publish("factory/line-1/#{sensorId}/temperature")
                  .qosAtLeastOnce()
                  .message(StringBody("{\"temp\":#{temp},\"ts\":#{timestamp}}"))
          ).pause(5)
      );

  {
    setUp(
        telemetry.injectOpen(
            constantConcurrentUsers(10000).during(Duration.ofMinutes(30))
        )
    ).protocols(mqttProtocol)
     .assertions(
         global().failedRequests().percent().lt(1.0)
     );
  }
}

A few things worth calling out:

• This uses constantConcurrentUsers, a closed workload model, on purpose. Unlike public web traffic, a device fleet has a fixed, known size, 10,000 sensors, not an open-ended arrival rate. Modeling it as 10,000 persistent connections that each publish on a loop is the more accurate representation of what's happening on the wire.
• qosAtLeastOnce() matters here. QoS 0 would understate the broker's real message-processing load; QoS 2 would overstate it unless that's genuinely what production uses. Test at the QoS level your fleet runs, not whichever is easiest to script.
• cleanSession(false) keeps the broker holding each device's subscription state across reconnects, which is realistic for most fleets and also the setting most likely to reveal a broker memory leak under sustained load, since session state accumulates rather than resetting.

To turn this into the connection-storm scenario that breaks brokers in production, run a second simulation that reconnects all 10,000 devices at once, simulating the aftermath of a regional network partition:

setUp(
    telemetry.injectOpen(
        atOnceUsers(10000)
    )
).protocols(mqttProtocol);

That single line is the entire difference between a test that looks like steady-state traffic and one that reproduces the failure mode that takes brokers down.

Two more MQTT-specific behaviors deserve testing beyond what fits cleanly in a code snippet. Last Will and Testament is configured at the protocol level so the broker publishes a predefined message if a device disconnects without a clean shutdown; load test it under a mass-disconnect scenario to confirm LWT messages don't themselves overwhelm downstream subscribers. Retained messages, which the broker holds and delivers to new subscribers immediately, should be checked for memory growth as retained-message volume increases across thousands of topics. Gatling's MQTT checks support both blocking (await) and non-blocking (expect) waits, plus a correlateBy mechanism for matching a publish to its corresponding acknowledgment or reply, which is what you'd use to measure real round-trip latency on a request-reply pattern rather than just confirming a publish succeeded.

Load testing thresholds that signal trouble

Watching a chart update in real time is useful. Knowing what number should make you worried is more useful, and it's the piece most IoT testing guides skip. A few anchors, some from protocol specs, some as a starting framework for setting your own:

MQTT QoS overhead is a spec fact, not a guess. QoS 0 is one packet per message. QoS 2 is four (PUBLISH, PUBREC, PUBREL, PUBCOMP). If your broker's message-per-second ceiling assumes QoS 0 traffic and your fleet runs QoS 2 in production, you're underestimating broker load by roughly 4x before a single device even reconnects.

CoAP's retry behavior is also spec-defined, in RFC 7252. The default confirmable-message parameters are ACK_TIMEOUT of 2 seconds, MAX_RETRANSMIT of 4 attempts, and NSTART of 1 outstanding request per endpoint. If you're testing anything upstream of a CoAP-to-MQTT gateway, these defaults tell you the retry storm shape you should expect from the device side during an outage, since a gateway that's slow to ack will trigger every downstream device's retransmit logic roughly on the same 2-second cadence.

For broker-side signals that don't come from a spec (queue depth, connection-accept latency, memory pressure), the right approach is to assert on the trend during a sustained run, not a single reading. A queue depth that's flat at 500 messages is fine. A queue depth that's climbing 50 messages per second with no sign of leveling off means your consumers can't keep pace with your producers, and it will hit a wall eventually, you just haven't run the test long enough yet to see where. Set your specific numeric ceilings based on your own broker's configured memory and your own consumer throughput, not a number borrowed from someone else's fleet.

Testing CoAP and AMQP

AMQP is genuinely testable with Gatling today, through the official community plugin, which also covers RabbitMQ. The testing considerations are different from MQTT's pub/sub model: AMQP's channel-based architecture means channel contention under concurrent load matters, along with unacknowledged message counts on a queue, which climb when consumers fall behind producers, and consumer prefetch count, which controls how many unacked messages a single consumer can hold and directly affects how quickly a backlog can form. Consult the plugin's own documentation for the exact request builders, since plugin APIs move independently of Gatling's core release cycle.

CoAP is the honest gap. Gatling doesn't ship a native CoAP protocol module. If your fleet talks CoAP directly, you have two practical paths: test at the gateway boundary, since most CoAP fleets terminate at an edge gateway that translates to MQTT or HTTP before reaching the cloud backend, and that boundary is fully testable with Gatling; or pair a dedicated CoAP tool for the device-facing leg with Gatling for everything on the cloud side. Don't let a tool's protocol gap talk you out of testing the constrained leg at all, just be clear about which tool is covering which hop.

Broker behavior under load

The article names HiveMQ, Eclipse Mosquitto, and AWS IoT Core as common MQTT brokers, and they behave differently enough under load that treating them as interchangeable is a mistake:

Your step-by-step testing checklist

• Before you start: document message sizes and protocol mix across your fleet, map expected network conditions (latency, jitter, loss), and define normal, peak, and burst traffic scenarios separately, since a single "expected load" number hides the burst behavior that causes most incidents.‍
• Running the test: ramp gradually through 10%, 100%, and 150% of expected load rather than jumping straight to peak; randomize connection offsets so your own test doesn't create an artificial burst that real traffic wouldn't produce; and validate the authentication path specifically, including TLS mutual auth and token expiry, since provisioning spikes have their own failure modes separate from steady-state traffic.‍
• Testing failure and recovery: inject packet loss and connectivity interruptions to confirm retry and backoff logic behaves the way it's supposed to under load, not just in a unit test; simulate broker and backend outages directly; and confirm devices use randomized backoff on reconnect, since synchronized backoff just delays the connection storm instead of preventing it.‍
• Monitoring: watch device-side and cloud-side metrics together so you can catch asymmetry (messages sent but not received, or vice versa); combine battery and power draw measurements with soak testing, since power issues often only surface after hours of sustained duty cycling; and where relevant, align your methodology with standards like ETSI TS 103 597/596.

Testing for both performance and security

Security and performance testing are usually run as separate exercises, on separate schedules, by separate teams. For IoT systems, that separation misses a real class of failure: vulnerabilities that only appear once a device is under load. A buffer handling routine that's fine at low traffic can fail exactly the way an attacker would want it to once queues start backing up.

A more integrated approach draws on a few existing frameworks rather than inventing new ones: OWASP's IoT security testing methodology and the ETSI EN 303 645 baseline requirements for MQTT and CoAP give a starting point for what to check; static and dynamic firmware analysis under stress can surface crash conditions that never show up in a functional test; and the Connectivity Standards Alliance's IoT Device Security Specification 1.0 provides a shared framework for treating performance and security compliance as one workflow instead of two. None of this replaces a dedicated penetration test, but running basic security checks inside your load test catches the specific subset of vulnerabilities that only exist under load.

What's coming next in IoT testing

A few trends worth watching, none of them fully mainstream yet: edge-aware testing frameworks built specifically for the distributed nature of edge computing rather than adapted from cloud-first tools; AI-driven anomaly detection that flags unusual performance patterns in complex fleets faster than a human reviewing dashboards; movement toward unified certification programs that test performance and security as one program rather than two separate audits; better battery-simulation fidelity, since this remains one of the harder things to model accurately outside of real hardware; and IoT-specific usability testing that accounts for how performance degradation feels to an end user, not just what a dashboard reports.

What this looks like at scale

InPost, the European parcel-locker and courier network, runs Gatling Enterprise across 60+ users and 30 teams, with a protocol stack that explicitly includes MQTT alongside gRPC, JDBC, and WebSockets for its Google Cloud, event-driven architecture. One of its most demanding validations is a 5-day continuous simulation mimicking a parcel's full journey from purchase to locker pickup, layering weekend backlogs on top of weekday surges, the kind of sustained, correlated load pattern that a short test simply can't reproduce.

"The five-day test revealed bottlenecks from code to cables. We tuned caches, optimized database indexing, even replaced physical connectors between servers and storage arrays."

Mateusz Piasta, Site Reliability Engineer, InPost

That test helped InPost scale from 1.6 million to over 10 million parcels a day.

Don't let your fleet become the 64%

IoT load testing isn't just a technical checkbox, it's the thing standing between "worked in the demo" and "held up during a regional outage with 50,000 devices reconnecting at once." The constraints are real: protocol diversity, duty-cycle behavior, and network conditions that a web load test never has to think about.

The practical path is combining approaches rather than picking one: protocol-level simulation for scale and iteration speed, real devices for a final authenticity check, and production traffic replay to validate that your test scenarios match what your fleet does in the wild. Whichever mix you choose, test at the QoS level, protocol, and workload model your fleet runs in production, not the ones that are easiest to script.