How to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AI
Your AI agent wrote a confident stock report saying 116% of analysts agree.
Nothing in your checks caught it. This is the workshop for fixing that. Laurie Voss (Head of Developer Relations at Arize AI, co-founder of npm) builds a full evals loop on a Claude-powered financial analyst agent: trace it with Arize Phoenix, read the traces, write code evals and LLM-as-a-judge evals, test whether the judge can be trusted, prove a prompt fix with experiments, upgrade from Claude Haiku 4.5 to 5.5 with evidence, and gate it all in CI with npm test. Speaker: Laurie Voss, Head of Developer Relations, Arize AI X: https://x.com/seldo https://arize.com Phoenix: https://phoenix.arize.com 0:00 Intro & setup 2:47 "Looks good to me" isn't a test suite 4:57 Traces vs.
The text is the source's own description of its publication. The content belongs to AI Engineer.
More from AI Engineer
Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca
AI Engineer 23 Oct
Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg
AI Engineer 23 Oct
Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens
Techmeme 4 h ago
Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount
Techmeme 6 h ago