Backtest Your Agent Against Yesterday's Web — Ryan Ahern, Exa
How do you know whether your agent would have made the right call?
Ryan Ahern of Exa shows how to use version-controlled search to see the web exactly as it was on a past date, then backtest an agent's predictions against what actually happened. The hands-on example predicts company acquisitions from signals like leadership turnover, then tunes the system without leaking future information. Chapters: - 00:00 Why search matters for agents - 04:15 Exa's search benchmark - 06:17 Versioning the whole web - 10:34 Avoiding look-ahead bias - 14:43 Backtesting and tuning a predictor - 18:49 Hands-on: predicting acquisitions - 22:54 Results and overfitting Ryan Ahern — https://exa.ai Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
Texten är källans egen beskrivning av publiceringen. Innehållet tillhör AI Engineer.
Mer från AI Engineer
Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca
AI Engineer 23 okt.
Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg
AI Engineer 23 okt.
Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens
Techmeme för 3 tim sedan
Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount
Techmeme för 5 tim sedan