AI performance testing culture: from traditional QA to intelligent testing

Diego Salinas
Enterprise Content Manager
Gatling
Table of contents
Add to Google preferred sources

Summarize this article with AI

Performance is everyone's job: AI is what makes that possible.

Performance testing has a scaling problem, and it isn't the load generator's fault.

Most engineering organizations still run performance testing the way they did a decade ago: one specialist, a queue of requests from other teams, and a report that lands in an inbox after the test is already over. That model doesn't break because the tooling is bad. It breaks because it depends on a small number of people who understand workloads, protocols, concurrency, and what a p99 spike actually means. When that person is busy, testing waits. When they leave, the tests often leave with them.

AI doesn't make individual load tests smarter. What it does is compress the specialist knowledge performance testing used to require into workflows any engineering team can run.

Performance testing is the mechanism. AI is the accessibility and feedback layer. Performance culture is the outcome.

Key takeaways

  • 01

    Performance testing has an ownership problem. Specialist-only testing creates queues, late feedback, and knowledge silos as engineering teams scale.

  • 02

    AI lowers the expertise barrier. It helps engineers create tests, migrate scripts, understand results, and investigate regressions.

  • 03

    AI systems change what performance means. LLM applications require teams to consider tokens, variable latency, inference capacity, retries, and cost.

  • 04

    Continuous workflows create the culture. Tests belong in Git and CI/CD, with shared objectives and results teams can understand.

  • 05

    The specialist does not disappear. Their role shifts from running every test to defining standards, reviewing results, and coaching teams.

Why the old model stalls at scale

Four patterns show up in almost every organization we talk to.

  • Siloed ownership creates a queue: when one team or one person owns every performance test, releases wait for their availability. That's fine at ten engineers. It's a bottleneck at a hundred. Intuit lived this directly: before its shift, load testing was a specialist-owned model that created late feedback, production regressions, and a queue for every team that needed a test.
  • Testing happens too late to matter: if performance validation runs once, right before a big release, regressions that crept in over the previous three months surface all at once, under deadline pressure, when they're 10–25× more expensive to fix.
  • Results don't travel: a load test produces percentiles, throughput, error rates, and injection profiles. That's a foreign language to the engineering manager who has to decide whether to ship. Before adopting run comparison tooling, Nickel's team felt this directly. "We used to export CSVs and graphs by hand," said Alexandre Baert, Pre-production Platform Manager. "Now we compare runs directly and get useful insights in real time."
  • Methodology drifts across teams: two teams running the same kind of test with different injection models, closed versus open, produce p95 numbers that aren't comparable. That quietly erodes trust in the data. Without a shared contract for what "pass" means, performance results turn into opinions instead of facts.

Mateusz Piasta, a Site Reliability Engineer at InPost, framed it well: "Observability is a culture here. If you want to ship code, you need to understand performance and how your services behave." That's a statement about how a team works together, not about which tool it uses.

How performance testing culture has evolved

The shift shows up in four consistent stages.

  • From waterfall testing to continuous integration: testing at the end of a long release cycle catches regressions after they're expensive. Testing on every commit or pull request catches them while they're cheap.
  • From specialist-driven to developer-owned: performance testing moves left to the people writing the code, cutting the handoff that used to sit between "I wrote this" and "someone else tells me if it's fast enough."
  • From manual scripts to test-as-code: version-controlled tests, reviewed in a pull request like any other code, don't rot silently the way a specialist's private script collection does. Ulrich Winter, Team Lead Infrastructure & Technology at EPI Company, chose Kotlin specifically because it's what his backend team already writes daily. "That really lowers the barrier for developers to contribute," he said.
  • From isolated runs to continuous intelligence: a single run's pass or fail tells you less than a trend across the last ten. An org that can see whether a critical flow is getting healthier or degrading over time is running a practice, not just a tool.

What AI changes

AI touches performance testing at five distinct points.

  • It generates tests from plain language: "simulate 5,000 users checking out, ramp over 10 minutes, keep p95 under 800ms" becomes a runnable test structure instead of hand-coded boilerplate. This doesn't remove the need to understand workloads and assertions. It removes the mechanical cost of turning that understanding into working code.
  • It interprets results in plain language: Gatling's AI Analysis works at three levels: a Run Summary after every test, Trend Analysis across recent runs, and Run Comparison across two to five runs. Each returns a verdict, a confidence level, and a specific recommendation instead of a raw chart.
  • It identifies root causes: pointing at the exact request or endpoint behind a regression is a different task than reporting that errors went up, and it's the difference between "the test failed" and knowing what to fix.
  • It migrates legacy tests: converting JMeter or LoadRunner scripts by hand is exactly the kind of work that keeps performance testing locked inside a central team, because rewriting hundreds of scripts is a project few teams can justify. AI-assisted conversion removes that specific barrier.
  • It orchestrates tests through coding agents: an MCP server and IDE assistants mean a developer can deploy, configure, and start a test from the terminal or their editor, in natural language, without switching to a separate tool.

Migration is a cultural opening as much as a technical one. Legacy performance programs typically look like a central team, GUI-heavy tooling, and periodic testing. AI-assisted conversion helps move that toward engineering teams, tests as code, and continuous feedback through CI/CD.

Loïc Chero, IT Leader at Tikamoon, made a similar bet without AI's help: his team rewrote its entire simulation suite in JavaScript specifically so the whole team could take ownership of it, not because the previous language was technically inadequate. "It's the most accessible language for our developers," he said, "and in e-commerce, we use it every day."

When AI is the system under test

As AI features move into production, applications gain a new kind of critical-path dependency, and that traffic behaves differently from a REST call.

Latency scales with output length, not just input size: a 500-token answer can take roughly ten times as long as a 50-token one. Concurrency is bounded by a finite inference pool instead of scaling cheaply on commodity CPUs, so exceeding it doesn't fail requests instantly. It queues them, and queuing on top of multi-second work produces a sharper latency cliff than a typical web tier ever sees.

Every request costs money, variably, tracking output length rather than a fixed per-request rate.

A load test built for a REST endpoint will pass cleanly against an AI-backed one and still miss what breaks in production: a latency tail that only appears once prompts vary in length, a retry storm when a bounded pool queues instead of failing fast and clients retry into the same scarce slots, or a cost curve that goes super-linear once average response length drifts. The right unit of throughput is tokens per second, not requests per second.

The right pass criterion is graceful degradation under load rather than zero failures, since a system that sheds load cleanly at capacity is healthier than one that queues everything and melts down. That's a large enough shift to deserve its own article rather than a paragraph here. Short version: AI-native systems need AI-native performance testing, not the same p95 assertion pointed at a different endpoint.

The four layers, put together

How AI changes performance testing AI • Performance culture
Layer What AI does What it changes culturally
Create Generates tests, migrates legacy scripts, and maintains simulations More engineers can get started, not just performance specialists
Understand Analyzes runs, explains regressions, and points to likely root causes Results become understandable to people who do not read percentiles every day
Test AI systems Load tests LLMs, agents, and AI-integrated applications Performance engineering expands to architectures that barely existed five years ago
Operationalize Connects test creation and analysis to Git, CI/CD, and observability Performance testing becomes continuous instead of a one-time pre-release activity

That fourth layer is the one that produces a culture. A team doesn't have a performance culture because it owns a load-testing tool. It has one when performance tests run on every pull request instead of once a quarter, when a regression is visible within minutes instead of after a postmortem, and when the specialist's job shifts from running every test personally to reviewing and coaching the teams who now run their own.

Intuit is the clearest large-scale example. In 2018, fewer than 100 developers used Gatling internally. Today more than 3,000 do, across 1,000+ engineering teams, running over 80,000 load tests a year, without a single specialist bottleneck. Getting there took specific decisions: performance test scaffolding shipped automatically with every new service repository, onboarding was automated through CI/CD, and templates were pre-built rather than written from scratch by every team. Chaitanya Bhatt, Principal Engineer at Intuit, framed the stakes plainly: "Any sort of outage means you're potentially disrupting someone's livelihood... We have no choice, reliability is not an option. It's a feature."

Common barriers, and what helps

Resistance to process change: teams comfortable with their current workflow won't adopt a new one because it exists. A quick, visible win on one flow or one pipeline does more than a mandate.

Lack of performance testing expertise: not every team has a dedicated performance engineer, and that used to be disqualifying. With 90% of developers now using AI, AI-assisted test generation and plain-language result summaries are specifically what lowers that floor.

Legacy tool dependencies: teams with years of JMeter or LoadRunner scripts have a real switching cost, not just inertia. AI-assisted migration changes the calculus here, because it addresses the actual cost (hundreds of scripts to rewrite) rather than just the preference for the old tool.

Unclear ownership: when performance is "everyone's responsibility," it can quietly become no one's. Explicit roles, from who authors tests to who reviews AI-generated ones, keep that ambiguity from turning into a gap.

Measuring whether it's working

Four metrics are worth tracking regardless of tooling: test coverage across critical paths, mean time to detect a regression after a code change, how many teams actively run tests rather than relying on one central group, and how many performance issues escape to production versus getting caught earlier.

A single pass or fail on one run doesn't tell you much. Gatling's Campaign feature scores every run of a test 0 to 100 across three categories: methodology (is the test well-configured, with a defined load type and at least one objective), objectives (did the run complete and meet its SLOs, weighted 60% of the score), and confidence (a weighted average of the last three runs, since a test that passes once and fails twice isn't reliable). Tracked daily, that turns "did it pass" into "is it getting better or worse," which is closer to what a Director of Engineering needs to answer before a release.

Building it, in order

  1. Start with one critical path, not everything: pick the flow closest to the business, checkout or login-to-dashboard, and get it under continuous testing before expanding.
  2. Put tests where developers already work: test-as-code in the application repo, reviewed like any other pull request, beats a separate tool only the specialist opens.
  3. Wire results into the pipeline, not an inbox: a performance regression should fail a build the way a broken unit test does. Tikamoon's rule is the clearest version of this: "No Gatling run, no production. It's not a suggestion. It's a rule."
  4. Make results legible to non-specialists: the people who run tests and the people who decide whether to ship are rarely the same people. AI-generated summaries and pass/fail thresholds exist specifically to close that gap.
  5. Track health as a trend, not a single pass or fail: Nicolas Zangari, QA lead at Purse, put the goal plainly: "The goal was to make testing a commodity, not an event where you have to mobilize people." A campaign score, checked daily, is what makes that possible.

Where this leaves the specialist

AI's contribution to performance engineering isn't a smarter individual test. It's a wider bench: more engineers who can write one, more managers who can read the result, and a specialist whose job moves from running every test personally to coaching the teams who now run their own.

{{card}}

About the author
Diego Salinas
Gatling

Diego Salinas Gardón is a senior technical copywriter and content strategist specializing in developer tools, SaaS, and software infrastructure. With hands-on experience in front-end development and modern web technologies.

He currently works at Gatling, where he creates content that helps developers and engineering teams better understand performance, testing, and modern software infrastructure.

FAQ

Can AI do performance testing?

AI generates tests from plain language, interprets results in readable summaries, identifies root causes of regressions, and migrates legacy scripts—but it doesn't replace the load generator or eliminate the need to understand workloads and assertions.

What is the 10/20-70 rule for AI?

This article doesn't cover the 10/20-70 rule for AI; it focuses on how AI compresses specialist knowledge in performance testing into workflows any engineering team can run, moving testing from a specialist bottleneck to a continuous practice owned by developers.

Will QA testers be replaced by AI?

AI shifts the specialist's role from running every test personally to coaching teams who now run their own tests—it widens the bench of engineers who can write and interpret performance tests rather than eliminating the need for performance expertise.

How do you measure AI performance?

AI-backed systems require different metrics than REST endpoints: tokens per second instead of requests per second, graceful degradation under load instead of zero failures, and monitoring for latency tails that appear when prompt lengths vary and inference pools queue requests.

Ready to move beyond local tests?

Start building a performance strategy that scales with your business.

Need technical references and tutorials?

Minimal features, for local use only