Stories by kkm

InfiniBand, RoCE, and All That

2678x Faster Matrix Multiplication with a GPU

UCCL-EP: DeepEP-style expert parallelism on any NIC, no GPU-initiated comms

Hacking Google with A.I. For $500k

How to setup a local coding agent on macOS

Anatomy of a high-performance EP kernel

No Token Left Behind: Demystifying Token-in-Token-Out in Miles

MoE expert co-activations: Reordering inputs yields easy throughput gains

The Economics of Speculative Decoding

Speculative KV coding: losslessly compressing KV cache by up to ~4×

70x faster cold(ish) starts for SGLang

Bringing Up DeepSeek-V4-Flash on AMD MI300X

Brave AI privacy:LLMs on NEAR AI Nvidia-Backed Trusted Execution Environments

How fast can an LLM go?

FHE can be leveraged for LLMs such as ChatGPT in a privacy-preserving manner

Harnessing the Power of Large Language Models for Insightful Review Analysis

A Privacy-First approach to use AI for understanding our Customers Better

How to make LLMs go fast

Leveraging Large Language Models for Sentiment Classification in Hotel Reviews

1 2 3 4 5 6