Introducing Index: Building The World’s Largest and Most Diverse Physical Dataset

This post details the technical architecture and operational scale of 'Index', Figure AI's proprietary pipeline for collecting and processing physical data for robot training. It describes the five-stage pipeline: automated filtering for quality, human-led fraud review, embedding-based deduplication for diversity, rebalancing using task quotas and clustering, and hierarchical text caption generation. The system handles 30 minutes of video uploads per second, requiring robust infrastructure for 24/7 availability and large-scale compute. The post highlights the challenges of sourcing diverse, real-world data at scale and the technical solutions implemented to overcome them.