Benchmark: Anthropic LLM models under load
How Claude models actually behave under real streaming load generated with Gatling.
Speed and price don't move together
The clearest signal in this benchmark isn't which model is fastest. It's that a premium-tier model can cost several times more per request without being proportionally faster. That gap only shows up under real load, not on a pricing page, which is the whole point of testing it directly instead of assuming it.
Performance by prompt size
Lower total time is better. Every model was sent the exact same three prompts (short, medium, long) under identical load conditions. TTFT and total time are shown as median / p95 across 20 requests per cell.
Note: Claude Sonnet 5's TTFT does not scale predictably with prompt size in this run. It's higher on short prompts (1,800 ms median) than on long ones (1,105 ms median), holding consistently at both median and p95. * On long prompts, Claude Sonnet 5 hit a voluntary output token cap set for this run on 8 of 20 requests, rather than finishing naturally — those are included in the Sonnet/long figures above. See methodology below.
| Model | Prompt size | Input tokens | Output tokens | TTFT (p50 / p95) | Total time (p50 / p95) | Cost / request |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | Short | 14 | 212 | 635 / 843 ms | 2,642 / 3,024 ms | $0.0011 |
| Claude Opus 4.8 | Short | 14 | 386 | 1,068 / 1,981 ms | 6,227 / 6,711 ms | $0.0097 |
| Claude Sonnet 5 | Short | 14 | 386 | 1,800 / 2,227 ms | 6,429 / 7,599 ms | $0.0039 |
| Claude Haiku 4.5 | Medium | 68–88 | 2,404 | 644 / 1,585 ms | 17,836 / 20,046 ms | $0.0121 |
| Claude Opus 4.8 | Medium | 68–88 | 1,405 | 1,162 / 1,603 ms | 21,054 / 23,203 ms | $0.0356 |
| Claude Sonnet 5 | Medium | 68–88 | 1,799 | 1,069 / 2,007 ms | 21,388 / 26,941 ms | $0.0182 |
| Claude Haiku 4.5 | Long | 315–437 | 2,894 | 2,100 / 7,460 ms | 32,484 / 37,958 ms | $0.0148 |
| Claude Opus 4.8 | Long | 315–437 | 2,690 | 1,117 / 1,804 ms | 45,508 / 50,139 ms | $0.0694 |
| Claude Sonnet 5 | Long | 315–437 | 3,627 | 1,105 / 1,391 ms | 45,881 / 53,076 ms * | $0.0371 * |
Throughput under load: short prompts
Higher tokens/sec is better. A simple one-line question, sent under identical concurrent load to every model.
Tokens generated per second, short prompts, under concurrent load.
Throughput under load: medium prompts
A multi-part technical question requiring a structured answer.
Tokens generated per second, medium prompts, under concurrent load.
Throughput under load: long prompts
A multi-paragraph diagnostic scenario, the kind of prompt a real production troubleshooting assistant would receive.
Tokens generated per second, long prompts, under concurrent load.
Speed and cost don't scale together
Lower is better. Real cost per request, long prompts: the widest gap in the whole benchmark.
Real $ cost per request (input + output), long prompts.
Ranking, by prompt size
Based on this load benchmark only. Response quality is a separate discipline and isn't measured here.
| Prompt size | Most cost-efficient | Highest throughput under load |
|---|---|---|
| Short prompt | Claude Haiku 4.5: $0.0011/req | Claude Haiku 4.5: 80.7 tok/s |
| Medium prompt | Claude Haiku 4.5: $0.0121/req | Claude Haiku 4.5: 133.6 tok/s |
| Long prompt | Claude Haiku 4.5: $0.0148/req | Claude Haiku 4.5: 88.4 tok/s |
Claude Haiku 4.5 leads throughput at every prompt size in this benchmark — a real result, not a rounding artifact. It's not the fastest to first token on long prompts, though: at 2,100 ms median TTFT (p95: 7,460 ms), it's the slowest of the three to start responding, even as it finishes fastest overall. Claude Sonnet 5 and Claude Opus 4.8 trade places behind it in throughput depending on prompt size (see the full breakdown table above).
How we benchmark
Every request went through Gatling's own SSE streaming client against Anthropic's live API, the same tool you'd use to load test your own AI feature, not a synthetic timer.
Each model received the exact same three prompts: short (~14 tokens), medium (~80 tokens), and long (~400 tokens), under an open workload model with 10 concurrent virtual users per model, each sending 6 requests in sequence, real streaming SSE connections, and no mocking. Time to first token and total generation time are captured separately, per the same distinction that makes AI load testing mechanically different from testing a traditional API: response time is not fixed, it's a function of how much the model decides to generate.
- 20 requests per model per prompt size (180 total), enough to treat percentile figures as meaningful rather than purely directional, though a larger run would still tighten the tails further.
- Every request's stop reason was captured and verified. This run used a fixed output token cap; on long prompts, Claude Sonnet 5 hit that cap on 8 of 20 requests, producing longer, costlier responses than a naturally-terminated one would. Nothing is hidden in the numbers above: 172/180 requests completed naturally (end_turn), 8/180 hit the cap — all Claude Sonnet 5, long prompts.
Your all-in-one load testing platform
Design complex tests, manage global infrastructure, and turn results into action on one powerful platform.
Need technical references and tutorials?
Minimal features, for local use only
