Benchmark: Anthropic LLM models under load
How Claude models actually behave under real streaming load generated with Gatling.
Speed and price don't move together
The clearest signal in this benchmark isn't which model is fastest. It's that a premium-tier model can cost several times more per request without being proportionally faster. That gap only shows up under real load, not on a pricing page, which is the whole point of testing it directly instead of assuming it.
Performance by prompt size
Lower total time is better. Every model was sent the exact same three prompts (short, medium, long) under identical load conditions.
| Model | Prompt size | Input tokens | Output tokens | TTFT | Total time | Cost / request |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | Short | 14 | 212 | 635 ms | 2,642 ms | $0.0011 |
| Claude Opus 4.8 | Short | 14 | 386 | 1,068 ms | 6,226 ms | $0.0097 |
| Claude Sonnet 5 | Short | 14 | 386 | 1,800 ms | 6,428 ms | $0.0039 |
| Claude Haiku 4.5 | Medium | 68–88 | 2,404 | 644 ms | 17,836 ms | $0.0121 |
| Claude Opus 4.8 | Medium | 68–88 | 1,405 | 1,162 ms | 21,054 ms | $0.0356 |
| Claude Sonnet 5 | Medium | 68–88 | 1,799 | 1,068 ms | 21,388 ms | $0.0182 |
| Claude Haiku 4.5 | Long | 315–437 | 2,894 | 2,100 ms | 32,484 ms | $0.0148 |
| Claude Opus 4.8 | Long | 315–437 | 2,690 | 1,116 ms | 45,508 ms | $0.0694 |
| Claude Sonnet 5 | Long | 315–437 | 3,627 | 1,104 ms | 45,881 ms | $0.0371 |
Speed under load: short prompts
Higher tokens/sec is better. A simple one-line question, sent under identical concurrent load to every model.
Tokens generated per second, short prompts, under concurrent load.
Speed under load: medium prompts
A multi-part technical question requiring a structured answer.
Tokens generated per second, medium prompts, under concurrent load.
Speed under load: long prompts
A multi-paragraph diagnostic scenario, the kind of prompt a real production troubleshooting assistant would receive.
Tokens generated per second, long prompts, under concurrent load.
Speed and cost don't scale together
Lower is better. Real cost per request, long prompts: the widest gap in the whole benchmark.
Real $ cost per request (input + output), long prompts.
Ranking, by prompt size
Based on this load benchmark only. Response quality is a separate discipline and isn't measured here.
| Prompt size | Most cost-efficient | Fastest under load |
|---|---|---|
| Short prompt | Claude Haiku 4.5: $0.0011/req | Claude Haiku 4.5: 80.7 tok/s |
| Medium prompt | Claude Haiku 4.5: $0.0121/req | Claude Haiku 4.5: 133.6 tok/s |
| Long prompt | Claude Haiku 4.5: $0.0148/req | Claude Haiku 4.5: 88.4 tok/s |
Claude Haiku 4.5 wins both categories at every prompt size in this benchmark, a real result, not a rounding artifact. Claude Sonnet 5 and Claude Opus 4.8 trade places behind it depending on prompt size (see the full breakdown table above).
How we benchmark
Every request went through Gatling's own SSE streaming client against Anthropic's live API, the same tool you'd use to load test your own AI feature, not a synthetic timer.
Each model received the exact same three prompts: short (~14 tokens), medium (~80 tokens), and long (~400 tokens), under an open workload model with 10 concurrent virtual users per model, each sending 6 requests in sequence, real streaming SSE connections, and no mocking. Time to first token and total generation time are captured separately, per the same distinction that makes AI load testing mechanically different from testing a traditional API: response time is not fixed, it's a function of how much the model decides to generate.
- 20 requests per model per prompt size (180 total), enough to treat percentile figures as meaningful rather than purely directional, though a larger run would still tighten the tails further.
- Every request's stop reason was captured and verified. No refusals, truncations, or anomalies are hidden in the numbers above.
Your all-in-one load testing platform
Design complex tests, manage global infrastructure, and turn results into action on one powerful platform.
Need technical references and tutorials?
Minimal features, for local use only
