From VLM/VLA's to Embodied Agents — Armen Aghajanyan, Perceptron AI

This post introduces Perceptron AI's approach to embodied agents, focusing on a perceptive objective that learns relevant visual tokens rather than treating all pixels equally. It addresses context bloat with data-sparse mixture of experts, allowing models to selectively process tokens. A key contribution is the discovery of new scaling laws where extensive video pretraining can substitute for significantly less teleoperation data, reducing training costs. The talk also previews an embodied foundation model trained on a petabyte of diverse data, capable of agentic tasks like detection and control, with an open release planned.