The Best Models Still Reason Like Toddlers — Andrew Dai, Elorian

Andrew Dai diagnoses that current frontier models excel at pattern matching but struggle with tasks requiring counting, spatial grounding, and state persistence, often failing a 'one-second test' for human-like reasoning. He highlights that popular benchmarks do not adequately measure these reasoning deficits. Elorian's approach is presented as a solution, comprising four key components: 1) development of visual reasoning datasets, 2) a synthetic data flywheel for evals, agents, SFT, and RL, 3) architectural changes to transformers, and 4) a native visual chain-of-thought mechanism where models explicitly ground their reasoning by drawing bounding boxes. The talk also touches upon applications in robotics, construction, and mechanical design, citing the limitations of current models in interpreting blueprints and CAD.