Stories by agcat

Ask HN: Anyone using Cloudflare Container platform in production?

Three-tier storage architecture to accelerate model loading for LLM Inference

AI Models Benchmarking for Education

Qwen2-7B-Instruct with TensorRT-LLM: consistently high tokens/SEC

LLM Wrapper Make Deployment with Nvidia Triton Inference Server Easier

Show HN: Open-source tool that writes Nvidia Triton Inference Glue code for you

Open Source CLI Tool to Generate Code for Nvidia Triton Deployment

Real-Time Streaming Apps with Nvidia Open Source Triton Inference

Fast Cold-starts for Serverless GPU Inference is becoming a reality

LLMs Tokens/Second Benchmark ( Mistral, Llama2, Gemma) – Independent Research

Show HN: Scale PDF Q&A App to 10K Users with GPUs – <$250/Mo

Finetune Phi-2 with DPO

Implement Fractional GPUs in Kubernetes to save upto 50% cost

Startup Pivots: How we Iterated on 4 Ideas in 12 weeks

Deploying Hugging Face Models on Nvidia Triton Inference Server at Scale

Have You Tried AWS Inferentia2 for ML Deployments?

Hugging Face X Inferless: First Ever AI After-Party in Bengaluru