State-Space Models for Vision Language Tasks

This post introduces a novel approach to training state-space models (SSMs) for vision-language tasks, specifically focusing on document intelligence for Indic languages. It details a multi-stage curriculum that prioritizes text token pre-training before visual data, a contrarian bet on block-level OCR with layout and reading order harnesses, and the use of verifiable reward reinforcement learning for OCR correctness. The resulting 3B parameter model demonstrates state-of-the-art performance, outperforming much larger transformer-based models on document AI benchmarks, and is being deployed in production for digitizing millions of pages.