Stories by zhisbug

Create a 5s 1080p Video in 4.5s with FastVideo on a Single GPU

Can LLMs play real-time games like supermario (other than Pokemon red)?

Sliding Tile Attention: A New Method That Speeds Up HunyuanVideo's Outputs by 3x

Fast Video Generation with Sliding Tile Attention

More Efficient Chain-of-Thought Reasoning Through Certainty Probing

AI Space Escape: Playing Games While Evaluting LLM Reasonsing

Efficient LLM Scheduling by Learning to Rank

FastVideo: a lightweight framework for accelerating large video diffusion models

MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Serving

Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x

Throughput Is Not All You Need: Maxing Goodput in LLM Serving via Disaggregation

Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

Important and MUST-KNOW techniques for a 2023 LLM serving system