SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind
In a panel discussion from Google DeepMind, researchers discuss challenges in evaluating AI-generated media.
When their video model recreated scenes from real videos, people often preferred the generated version — but this was because it was sharper and more color-saturated, not more realistic. The panel highlights several difficulties: language is a poor intermediate representation for things like sound, taste, and skin tones that humans are sensitive to, AI video models always sound studio-quality because that's what they're trained on, and models exploit rewards by adding wedding rings without anyone noticing during development. Finally, they conclude that evaluation of video models still must be done manually — ten people watching two videos and choosing one.
Vibekollen prepared this summary with AI from the original publication. The content belongs to AI Engineer.
