Building AI Agents for AR Glasses and XR Devices with NVIDIA XR AI | NVIDIA Technical Blog

What this post added

Introduces NVIDIA XR AI, an open-source library for building intelligent agents for AR glasses and XR devices. This framework enables real-time visual and voice interaction, enterprise data access, and tool integration by connecting XR devices to GPU-accelerated AI services. The architecture is modular, separating components like media transport, model services (including NVIDIA Cosmos for vision and Nemotron for language), enterprise connectivity via Model Context Protocol (MCP), and agent orchestration using frameworks such as NVIDIA NeMo Agent Toolkit, supporting flexible deployment and multi-user scenarios. Developers can quickly prototype XR agents by cloning the public beta repository, running sample agents, integrating enterprise data through MCP servers, adding agent orchestration, and optionally incorporating CloudXR for rendered spatial experiences.