Benchmarks
Speed, latency and accuracy we measured ourselves on October 31, 2026, on the GPUs we tested for serving each model. The method is at the foot of the page.
MiMo-V2.5 on 2x NVIDIA B200
Official FP8 weights, split across both GPUs (tensor parallel 2), vLLM 0.32.
Quality check
Before load testing, the same server answered two objective sets, with reasoning on and the model's own tool-call parser.
- 40 of 40
- Math: GSM8K test problems, scored on the final number
- 30 of 30
- Tool calls: tool-calling cases, scored on calling the right function with the right argument values
Under load
Long agent-style conversations of about 40,000 input tokens per request.
| Sessions | Requests/s | Tokens/s per user | TTFT p50 | TTFT p95 | Cache hits | Input tokens, avg |
|---|---|---|---|---|---|---|
| 8 sessions | 0.99 | 136 | 2.89 s | 5.66 s | 82% | 38,014 |
| 16 sessions | 1.34 | 96 | 4.22 s | 6.13 s | 86% | 42,994 |
At 32 sessions the cache hit rate fell to 30% and the median time to first token rose to 14.31 s, so a MiMo-V2.5 server takes at most 16 sessions and answers busy past that.
Qwen3.6-35B-A3B on one GPU
Each card at the loads where it stays fast.
Under load, on two cards
On the NVIDIA B300, Qwen's official FP8 weights; on the NVIDIA RTX PRO 6000, NVIDIA's FP4 (NVFP4) build of the same model. Agent-style conversations of about 12,500 input tokens per request on both.
| GPU | Sessions | Requests/s | Tokens/s per user | TTFT p50 | TTFT p95 | Cache hits | Input tokens, avg |
|---|---|---|---|---|---|---|---|
| NVIDIA B300, FP8, 16 sessions | 16 | 5.74 | 159 | 0.52 s | 0.74 s | 77% | 12,203 |
| NVIDIA B300, FP8, 32 sessions | 32 | 8.37 | 117 | 0.71 s | 1.19 s | 78% | 12,956 |
| NVIDIA RTX PRO 6000, FP4, 8 sessions | 8 | 2.22 | 121 | 0.69 s | 0.86 s | 77% | 12,422 |
| NVIDIA RTX PRO 6000, FP4, 16 sessions | 16 | 2.99 | 81 | 1.05 s | 1.47 s | 78% | 13,246 |
Speed as load rises
Every load we ran, including the ones past where we stop.
Output tokens per second, per user
The median speed one stream receives after its first token. MiMo-V2.5 requests carry about three times the input of Qwen3.6-35B-A3B requests, so read each line against itself as load rises, not against the others.
- MiMo-V2.5, 2x NVIDIA B200 FP8
- Qwen3.6-35B-A3B, NVIDIA B300 FP8
- Qwen3.6-35B-A3B, NVIDIA RTX PRO 6000 FP4
How we measured
Every number on this page comes from these runs.
- Where
- Rented cloud GPU servers, with the model server (vLLM) and our load generator on the same machine. The times leave out the internet: through the API, add your network round trip and our gateway.
- Traffic
- Streaming chat requests from many concurrent sessions, each a multi-turn conversation that re-sends its whole history, the way agents and chat apps do. Prompt text is cut from public-domain books with each model's own tokenizer, so token counts are exact.
- Each load
- A fixed number of concurrent sessions for about two minutes. The first 15% of each run is dropped as warm-up. Output length is fixed per request so every run does the same work. No request failed at any load on this page.
- Requests/s
- Completed requests per second across all sessions.
- Tokens/s per user
- The median output speed of a single stream, counted from its first token to its last.
- TTFT
- Time to first token: from sending a request to receiving its first streamed token. p50 is the median, p95 the time 95% of requests beat.
- Cache hits
- The share of input tokens the server answered from its prefix cache, read from vLLM's own counters. Cached tokens are what the cached input price applies to.
- Quality sets
- 40 problems from the GSM8K test set, scored on the final number, and 30 tool-calling cases across four tools, scored on calling the right function with the right argument values.
- Ask for the raw data
- Every run wrote one line of results per load. Email [email protected] and we will send them.
MiMo-V2.5 request shape
Multi-turn sessions of up to 10 requests: a 4,000-token system prompt shared by every session (5 variants), a 16,000-token first message, then 4,000 new tokens and the previous answer added each turn and the whole conversation re-sent. 779 output tokens per request. About 40,000 input tokens per request on average.
Qwen3.6-35B-A3B request shape
Multi-turn sessions of up to 8 requests: a 2,500-token system prompt shared by every session (5 variants), a 1,500-token first message, then 2,000 new tokens and the previous answer added each turn and the whole conversation re-sent. 357 output tokens per request. About 12,500 input tokens per request on average.