Writing
The thesis, the results, and the engineering discipline behind them — measured, cited, and reproduced across three serving stacks and two GPU vendors.
In KV-bound serving, bytes per resident token shape both the memory ceiling and the attention-traffic floor — here is the model, the measurements, and the validation protocol.
Modern LLM serving is throttled by memory bandwidth and capacity, not arithmetic — so the one number that moves the wall is the per-token cache footprint, b.
A ground-up port to GB10, gated on an fp32-referenced numerical check at every milestone, with the latency close told as a roofline-and-kernel-engineering story — a 2.85× faster decode kernel, τ ≈ 1.07 at 3.4× capacity.
What it takes to trust a serving-layer change in production — a correctness-first gate, capture-replay bisection, statistical validation, and one-flag fault isolation — on a live 8×MI350X deployment.