Skip to content
VibekollenBETAVibekollen
VideoAI Engineer

Beyond Static Intelligence: Evaluating Continual Learning — Parth Asawa, UC Berkeley

Parth Asawa from UC Berkeley criticizes how AI models are currently evaluated.

All ranking lists rely on testing a model on one task, resetting its memory, and testing the next task — which hides how well the system actually learns over time. He presents a new measurement tool that compares the same system when it can learn (retain memory between tasks) versus when it is reset after each task. The results are surprising: simple in-context learning (where the model uses previous answers to improve itself within the same conversation) outperforms more complex memory management systems. His benchmark covers six areas, from database search to forecasting models, and reveals problems such as models forgetting their own corrections.

Open on YouTube →

Vibekollen prepared this summary with AI from the original publication. The content belongs to AI Engineer.

More to read