Stories by somnial

The case for disaggregated LLM serving

On-the-fly snapshot compression for elastic inference at scale

NVLink, NVSwitch, and All That

The Anatomy of an Instruction Pipeline Hazard

Width vs. Depth: Speculating on the Margin

Pushing memory bound CUDA kernels past the speed of light with data compression

Speculative KV coding: ~4× losslessly compressed KV cache using a small model

70x faster cold(ish) starts for SGLang

LLM powered data structures: A lock-free binary search tree

Parallel Primitives for Multi-Agent Workflows

Scheduling in LLM Inference

How fast can an LLM go?