Why AI Goes Rogue: Turing Award Winner Explains Reward Failure | Andrew Barto
Artikelbild eller återanvändbart omslag för Eye on AI
Andrew Barto on Rewards Gone Rogue Andrew Barto helped found reinforcement learning.
He and Richard Sutton wrote the field's standard textbook, and they shared the Turing Award for that work. Reinforcement learning now trains today's reasoning models and AI agents. It also sits behind some of their strangest behavior, including the recent cases of agents breaking out of their sandboxes. Craig visited Barto at his home on Cape Cod. They talked about where rewards come from and why they are so hard to get right. Barto traces the problem back to Norbert Wiener's warnings in the 1950s. He explains how the dopamine system in the brain mirrors the learning algorithm he and Sutton developed. And he admits a weakness in their own framework: boiling every goal down to a single number. He also makes the case for engineering discipline. Ship designers learned their codes from wrecks and drowned sailors. Barto hopes AI can learn its lessons without that kind of cost. Subscribe to Eye on A.I. for weekly conversations with the people building and deploying the future of AI.
Texten är källans egen beskrivning av publiceringen. Innehållet tillhör Eye on AI.
Mer att läsa
Experts say AI has made planning cyberattacks on industrial equipment faster by helping attackers read manuals and research past accidents more quickly
Techmeme för 1 tim sedan
Current and former officials from the US and other countries describe the Chinese spy agency MSS' campaign to help Chinese companies acquire advanced AI chips
Techmeme för 5 tim sedan
Tesla rolls out FSD v15, featuring a photorealistic rendering of car surroundings; Elon Musk says it is real-time AI video generation without using game engines
Techmeme för 6 tim sedan
Sources: AI server suppliers like Dell, HPE, Lenovo, and Supermicro are demanding substantially higher prices from buyers, months after contracts were signed
Techmeme för 7 tim sedan
