Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain

Vivek Trivedi from LangChain argues that improving AI agents is about analyzing trace data — the logged steps and decisions an agent takes while working.

Instead of guessing why an agent performs worse after updates, you can have other AI models read through these traces and identify problems. Trivedi contends that observability and continuous learning are the same problem viewed from different angles. He demonstrates that cheaper, open-source models can achieve the same quality as expensive top-tier models if you're smart about how you phrase instructions — and that it pays to iterate on instructions until progress slows, then fine-tune the model to advance further, then return to refining instructions again.

More from AI Engineer

Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca

AI Engineer · 23 Oct

Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg

AI Engineer · 23 Oct

Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens

Techmeme · 4 h ago

Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount

Techmeme · 6 h ago