TechPrufer
Get started
NVIDIA
NVIDIA logoNVIDIA · Feature trail

Omniverse NuRec for AV Perception Adaptation

1 post · Jun 2026 – Aug 2026

What this trail covers

This feature thread tracks the evolution of NVIDIA Omniverse NuRec for reconstructing real-world driving scenes and rendering new camera views for target vehicle configurations. Initial efforts focused on enabling novel-view synthesis and reusing scene data for adaptation. Subsequent developments have packaged the workflow as agent skills for downloading scenes, rendering with NuRec, and applying Harmonizer for temporal consistency. This post details the process of adapting perception models to specific vehicle configurations and optimizing the reconstruction pipeline using NVIDIA Nsight Developer Tools. Optimizations focused on kernel fusion, reducing synchronization, and improving GPU occupancy, leading to significant runtime reductions in the neural reconstruction pipeline.

All NVIDIA trails
  1. Scale AV Perception Across Vehicle Platforms with NVIDIA Omniverse NuRec | NVIDIA Technical BlogThis post details a practical workflow for adapting autonomous vehicle perception models to new vehicle platforms using NVIDIA Omniverse NuRec. It outlines a four-step process: downloading reconstructed scenes from the Physical AI NuRec Dataset, rendering these scenes from a target vehicle's sensor rig configuration (demonstrating novel-view synthesis with customizable camera parameters), refining the rendered frames with NVIDIA Harmonizer for temporal consistency, and finally, training perception models on the generated synthetic data. The post highlights the benefits of this approach for carline adaptation, such as reducing the need for extensive real-world data collection early in development, and provides code snippets for scene download and rendering using the NuRec container.
  2. Optimizing a Neural Reconstruction Pipeline Using NVIDIA Nsight Developer Tools | NVIDIA Technical BlogThis post details the optimization of the NuRec neural reconstruction pipeline using NVIDIA Nsight Systems and Nsight Compute. Key contributions include identifying performance bottlenecks in the `interpolate` function, fusing small kernels to achieve a ~50x speedup, and removing unnecessary `cudaStreamSynchronize` calls to improve GPU utilization. The analysis also revealed opportunities for further optimization by addressing workload imbalance and long-tail execution effects through warp activity analysis.