How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase

Evan Kormos, a player-coach at Coinbase with 15 years in enterprise financial services, lays out a practical way to bring agents into policy-driven, regulated workflows.

He models processes as DAGs and loops, explains why a loop is only as trustworthy as its check, and shows how evals and traces become a new confidence mechanism. Agents then move through three stages: fully supervised, then sampled, then humans handling only exceptions. Chapters: - 00:00 Policy is the original system prompt - 02:53 Where agents fit: people, process, technology - 03:49 Workflows as DAGs - 05:31 Inside a node: the loop - 07:16 Exception flows and bottlenecks - 08:59 Quality inputs, quality outputs - 10:43 The loop is only as good as the check - 11:47 Evals as a confidence mechanism - 12:39 Which steps should an agent own? - 14:29 Supervised, then sampled, then exceptions only - 15:31 Takeaways Evan Kormos — https://www.coinbase.com Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!

Mer från AI Engineer

Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca

AI Engineer · 23 okt.

Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg

AI Engineer · 23 okt.

Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens

Techmeme · för 3 tim sedan

Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount

Techmeme · för 5 tim sedan