The results
33 results. Each is compared against the model's own standard setup on the same hardware.
| Area | Result | What we measured | Compared to | Setup |
|---|---|---|---|---|
| Speed | 1.60× | Tokens per second with 4 users: 255 vs 159 | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | 1.43× | Tokens per second with 32 users: 1,033 vs 722 | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | ≥2.1× | Tokens per second with 128 users: 1,926 vs 905The standard stack could run only 54 of the 128 users. Yantrion's 1,926 is a conservative floor. | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | 1.44–1.62× | Speed with 28K tokens of context, 17 usersIn August this same test ran 1.48× slower. We found the cause and fixed it. | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | 1.72× | Speed with 128K tokens of context, 4 users | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | 2.32× | Speed with 256K tokens of context, 4 users | Standard SGLang, same GPUs | Kimi-K3, AMD Instinct MI350X, SGLang |
| Speed | within ~5% | Our standard-stack result (905 tokens/s per machine) vs independent published results for the same model on AMD MI355X (952)A fair baseline, not a weak one. | Wafer, July 2026 | Kimi-K3 |
| Capacity | 2.54× | Users at once in the same GPU memory: 193 vs 76Measured on a fixed set of real sessions, identical for both sides. | The model's own standard memory format | Qwen3.8 |
| Capacity | 2.00× | Users at once in the same GPU memory: 32 vs 16 | The model's own standard memory format | Kimi-K3 |
| Capacity | +53% | Tokens held in exactly the same GPU memory: 1,964,288 → 3,013,376 | The model's own standard memory format | DeepSeek-V4-Flash |
| Capacity | 1.9× | Memory made smaller | The model's own standard memory format | GLM-5.3 |
| Capacity | 4.49× | Memory made smaller on the parts that can shrink; about 1.6× for all memory at 32K tokensSimulated on the real model, not yet run live. | The model's own standard memory format | GPT-OSS-120B |
| Capacity | 3.97× | Memory for the model's recurrent layers made smaller3.74× across the whole memory pool. | The model's own standard memory format | Nemotron 3 Ultra, NVIDIA H200 |
| Reach | 1,048,576 | Tokens in a single prompt, with 128 of 128 identical answers on our strictest check | The standard stack stops at 307,571 tokens on the same setup | Kimi-K3 |
| Reach | 4 GPUs | A 1M-token context served on four NVIDIA H200sWith compact 4-bit model weights. | Normally needs eight | Nemotron 3 Ultra, NVIDIA H200 |
| Agents | 150/150 | Agents resumed with memory intact after the whole fleet was cycled | Best standard setup that would run: 0 of 150, same machine, same day | 150 agents, 36K tokens each |
| Agents | 0 failures | Failed requests while the fleet grew from 150 to 300 agents | Slows down gracefully, never breaks | Agent fleet, 36K tokens each |
| Agents | 12.7 s | Typical time to resume an agent with 150 running, vs 28.5 s starting over. With 96: 1.7 s vs 33.2 s. | Starting over | Agent fleet, 36K tokens each |
| Agents | 580 | Saved agent sessions per machine | 116 without Yantrion | Nemotron 3 Ultra, NVIDIA H200 |
| Quality | 128/128 | Multi-step agent conversations with word-for-word identical answers | The standard stack, no partial credit | Both test campaigns |
| Quality | Pass | Needle-in-a-haystack long-document recall | Hidden fact found | Kimi-K3, AMD Instinct MI350X, SGLang |
| Quality | 15/16 | GSM8K math reasoning | Problems solved | Kimi-K3, AMD Instinct MI350X, SGLang |
| Quality | 195 | Test runs saved and fingerprinted | Every run kept on record | Both test campaigns |
| Memory | 1.696× | Memory set aside, still ready to reuse | The model's own standard memory format | Nemotron 3 Ultra, NVIDIA H200 |
| Memory | 3.015× | Memory set aside, still ready to reuse | The model's own standard memory format | DeepSeek-V4 |
| Memory | 1.89–2.14× | Memory sealed and saved, portable between machines | The model's own standard memory format | Saved memory |
| Memory | 3.664× | Smaller to send over the networkCounted on its own, never added to memory savings. | Saved memory before sending | Transfer |
| Ecosystem | SGLang | High-speed open-source serving engine | Proven end to end with real models | Supported and qualified |
| Ecosystem | vLLM | Widely used open-source serving engine | Proven end to end with real models | Supported and qualified |
| Ecosystem | TensorRT-LLM | NVIDIA's high-performance serving engine | Proven end to end with real models | Supported and qualified |
| Ecosystem | NVIDIA Dynamo | AI serving at data-center scale | Proven end to end with real models | Supported and qualified |
| Ecosystem | NIXL | Fast transfer of AI memory between GPUs and machines | Proven end to end with real models | Supported and qualified |
| Ecosystem | LMCache | Stores and reuses AI memory | Proven end to end with real models | Supported and qualified |
How we compare
The rules every number follows.
- Every comparison is against the model's own standard setup on the same hardware, never an easy baseline.
- Nothing is compared unless we measured the standard stack on the exact same test.
- We never mix different kinds of measurement in one comparison.
- Savings on storage and transfer are reported on their own, never added to memory savings.
- No speed tricks like speculative decoding are counted. Those gains would come on top.
- We never claim to be number one. Other benchmarks measure things too differently to rank fairly.
What our checks threw out
Good-looking results that didn't hold up.
- A compression setting that changed the model's answers. Thrown out.
- A shortcut that saved more memory but hurt quality. Thrown out.
- A test run where Yantrion never actually switched on. Discarded, because it can't count for us.
- A faster 32-user setup that failed the quality check. We publish the slower result that passed: 30.045 ms per step.
- Our own August result, 1.48× slower than the standard stack. We said so, found the cause, and fixed it. The same test now runs 1.44–1.62× faster.
Where we're still growing
Stated plainly.
- If every agent is paused and resumed at the very same moment, 54 of 150 come back with memory intact today. Our next memory tier is designed to close that gap.
- For a single user, speed isn't yet where our many-user results are. The biggest wins show up when many people use a model at once.
- GPT-OSS-120B results are simulated on the real model, not yet run live.
Quality comes first
A result only counts if the model gives the same answers. No partial credit, and every run is kept on record.
128/128
Same answers on agent tasks
Multi-step agent conversations matched word for word. No partial credit.
Pass
Long-document recall
Needle-in-a-haystack tests: the hidden fact is found.
15/16
Math reasoning
GSM8K grade-school math problems.
195
Runs on record
Every test run saved and fingerprinted.