Reinforcement Learning without Verifiable Rewards — Will Brown, Prime Intellect

Reinforcement learning has been easy to sell where the answer is checkable, like math or code, and Will Brown's talk is about everything else.

Most valuable tasks have no clean verifier, so Prime Intellect's work is on how you build reward signal when there is no ground truth waiting. He frames RL simply first, a model acting in a harness with tools and skills, getting a reward, and nudging its weights, then asks how you keep climbing once you leave the verifiable island behind. His answer leans on environments as the anchor. You can set up judges, generate question and answer pairs grounded in real documents and repos, and use a reverse direction trick where you hide something, like a bug or a backdoor, so the model can learn to find it again, which conveniently gives you a difficulty dial to keep tasks not too easy and not too hard. He is direct about the dangers: reward hacking will find you if you are not careful, so you inspect traces, run small experiments, and bring in expert understanding. The goal he keeps returning to is making this a real science, with open models and shared benchmarks, where environments turn into new tasks and higher levels of ability.

Mer från AI Engineer

Inside Arca's Long-Horizon Agents That Run the Business - Aman Singh, Arca

AI Engineer · 23 okt.

Real AI Agent Attacks and What Would Have Stopped Them — Andrey Rybka, Bloomberg

AI Engineer · 23 okt.

Outside of tech there has been no financial payoff to US companies for using AI, now viewed as a margin-hurting cost center, as VC-backed firms burn more tokens

Techmeme · för 3 tim sedan

Source: Replit recently moved most AI spend from Anthropic to OpenAI, which gave 30%+ off in a deal worth $100M+/year, after Anthropic discontinued its discount

Techmeme · för 5 tim sedan