Holotron-12B - High Throughput Computer Use Agent

This post introduces Holotron-12B, a multimodal computer-use model built on NVIDIA's Nemotron architecture. It highlights the model's hybrid State-Space Model (SSM) and attention mechanism, which optimizes for high-throughput serving and long-context inference by avoiding the quadratic cost of full attention. The post details the model's training process, its performance improvements on agent benchmarks (WebVoyager, OS-World-G, GroundUI, WebClick) compared to the Nemotron base model and Holo2-8B, and its inference efficiency gains (over 2x higher throughput on WebVoyager with vLLM optimizations). It also discusses the model's ability to scale efficiently with increasing concurrency, achieving 8.9k tokens/s at 100 concurrent workers.