Blogs›Physical Intelligence›Real-Time Action Chunking
The FAST tokenizer, inspired by continuous compression methods like JPEG, compresses raw robot action chunks using Discrete Cosine Transform (DCT) and Byte Pair Encoding (BPE). This allows for a 10x compression of action sequences into dense tokens, enabling faster training (5x) of autoregressive transformer policies for dexterous robotic tasks, achieving similar dexterity to diffusion models. FAST enables generalist policies to zero-shot generalize to new environments and instructions, as demonstrated with the DROID dataset.
6/9/2025
Introduced the Real-Time Action Chunking (RTC) algorithm, which adapts inpainting techniques from diffusion and flow models to ensure consistency between action chunks in VLA models. RTC allows for real-time execution without training-time changes, significantly speeding up inference and improving robustness to latency.
2/26/2025
Introduces the 'Hi Robot' system, a hierarchical inference process for VLA models. This system uses a high-level VLM ('System 2') to reason through complex prompts and language, generating intermediate steps for a low-level VLA model ('System 1'). The high-level policy is trained using synthetic data and can incorporate real-time user feedback. The post presents quantitative results showing improved instruction-following accuracy and task progress compared to flat VLA policies and GPT-4o.
1/16/2025
Introduced the FAST (Efficient Robot Action Tokenization) method, which uses Discrete Cosine Transform (DCT) combined with Byte Pair Encoding (BPE) to compress robot action sequences into discrete tokens. This approach significantly improves training efficiency (5x faster) and enables autoregressive transformer models to handle highly dexterous tasks with precision, achieving performance comparable to diffusion models while allowing for faster training and better generalization.