Omni-bodied Learning from Video
Skild AI's Omni-bodied Learning enables robots to acquire new skills by watching human videos, bridging the embodiment gap and overcoming the data bottleneck in robotics. This approach leverages abundant human video data and requires minimal robot-specific data for fine-tuning. This post details the development of an omni-bodied brain trained on a universe of 100,000 different robots, enabling end-to-end locomotion driven entirely by online vision and proprioception. The model demonstrates in-context learning, adapting to novel scenarios such as limb loss, broken legs, jammed wheels, and walking on stilts, showcasing resilience and zero-shot control capabilities.