Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod
Most teams spin up a new open model with the default vLLM recipe and hope for the best.
Runpod forward-deployed engineer Chris Vela shows how to choose configs from data instead, starting with the use case and concurrency, and demos BenchPod, a benchmarking dashboard built on vLLM that tests configs live without restarts. He runs Kimi K3 on a B300, shows how tuning max batch tokens cuts inter-token latency, and asks whether you need a frontier model at all. Chapters: 0:00 Intro 0:29 What Runpod does 1:32 Everyone's guessing at vLLM configs 2:53 Version and CUDA gotchas 3:48 BenchPod: benchmarking on top of vLLM 6:07 Running Kimi K3 on a B300 7:40 Demo 10:01 Tuning max batch tokens 13:34 From R&D to production 14:21 Do you need the frontier model? Chris Vela https://www.runpod.io Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
The text is the source's own description of its publication. The content belongs to AI Engineer.
More to read
Lumaril, which develops AI software for optimizing industrial production by connecting to existing control systems, raised a $66M Series A at a $400M valuation
Techmeme 3 h ago
DeepSeek open sources Super-Agent as a research prototype, a proactive coding agent it says can monitor group chats for coding issues and suggest solutions
Techmeme 4 h ago
Insurance software maker Guidewire agrees to acquire Hyperexponential, which makes AI-powered insurance underwriting and pricing software, for $1.7B in cash
Techmeme 4 h ago
One word for AI agents in 2027
Nate Herk 8 h ago