Is a Faster Model Actually Faster? Test the Whole Job — Morgan Suo, MiniMax

A faster model only helps if the finished job gets faster. Morgan Suo of MiniMax uses one real research task to test the changes inside the M3.1 Flash model: an analyst checking whether circuit-board suppliers are turning AI demand into cash.

The changes are sparse attention on long inputs, a 4-bit KV cache, low-precision experts, speculative decoding, server load and reasoning budgets. Each one saves compute, and each can quietly cost accuracy or wait time. She then shows how to judge a model switch: check sources and calculations, count the full cost of a usable report, and make the result repeatable. Chapters: - 00:00 Intro - 01:44 The research job - 03:20 Profit vs. cash flow - 04:22 What changed inside the model - 05:09 Sparse attention on long inputs - 07:53 4-bit KV cache - 08:59 Low-precision experts - 10:20 Speculative decoding - 11:39 Throughput vs. wait time - 13:09 Reasoning budgets - 14:41 How to evaluate a model switch - 16:12 Recap Morgan Suo, MiniMax: https://www.minimax.io Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!

Mer från AI Engineer

Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca

AI Engineer · 23 okt.

Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg

AI Engineer · 23 okt.

Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens

Techmeme · för 3 tim sedan

Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount

Techmeme · för 5 tim sedan