How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase
Evan Kormos, a player-coach at Coinbase with 15 years in enterprise financial services, lays out a practical way to bring agents into policy-driven, regulated workflows.
He models processes as DAGs and loops, explains why a loop is only as trustworthy as its check, and shows how evals and traces become a new confidence mechanism. Agents then move through three stages: fully supervised, then sampled, then humans handling only exceptions. Chapters: - 00:00 Policy is the original system prompt - 02:53 Where agents fit: people, process, technology - 03:49 Workflows as DAGs - 05:31 Inside a node: the loop - 07:16 Exception flows and bottlenecks - 08:59 Quality inputs, quality outputs - 10:43 The loop is only as good as the check - 11:47 Evals as a confidence mechanism - 12:39 Which steps should an agent own? - 14:29 Supervised, then sampled, then exceptions only - 15:31 Takeaways Evan Kormos — https://www.coinbase.com Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
The text is the source's own description of its publication. The content belongs to AI Engineer.
More from AI Engineer
Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca
AI Engineer 23 Oct
Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg
AI Engineer 23 Oct
Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens
Techmeme 4 h ago
Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount
Techmeme 6 h ago