TechPrufer
Get started
Lambda
Lambda logoLambda · Feature trail

Single-Image 3D Scene Reconstruction

3 posts · Jul 2026 – Sep 2026

What this trail covers

The platform now supports PixARMesh, a novel method for single-image 3D scene reconstruction. This approach tokenizes image, object pose, and mesh into a single sequence, allowing an autoregressive Transformer to predict object poses and shapes token by token in one forward pass. It directly generates artist-ready meshes without relying on implicit representations or multi-stage pipelines, improving reconstruction quality, especially under occlusion. PixARMesh sets a new state of the art on the 3D benchmarks. This post introduces CVP (Central-Peripheral Vision-Inspired Multimodal Model), a novel approach for spatial reasoning in AI. CVP combines a target-affinity token for focusing on task-relevant objects with an allocentric grid for global spatial awareness, inspired by human vision. It achieves state-of-the-art results on 3D vision-language benchmarks like ScanQA, SQA3D, ScanRefer, Multi3DRefer, and Scan2Cap, demonstrating improved performance in 3D question answering, visual grounding, and dense captioning. The post highlights the importance of representation in spatial reasoning and showcases Lambda's GPU infrastructure for training such compute-intensive multimodal workloads.

All Lambda trails
  1. Building multimodal models for spatial reasoningThis post introduces CVP (Central-Peripheral Vision-Inspired Multimodal Model), a novel approach for spatial reasoning in AI. CVP combines a target-affinity token for focusing on task-relevant objects with an allocentric grid for global spatial awareness, inspired by human vision. It achieves state-of-the-art results on 3D vision-language benchmarks like ScanQA, SQA3D, ScanRefer, Multi3DRefer, and Scan2Cap, demonstrating improved performance in 3D question answering, visual grounding, and dense captioning. The post highlights the importance of representation in spatial reasoning and showcases Lambda's GPU infrastructure for training such compute-intensive multimodal workloads.
  2. Giving robots 3D vision without depth sensorsThis post introduces StereoPolicy, a system that enables robot policies to learn 3D spatial understanding directly from stereo images. It replaces single-camera observations with synchronized stereo image pairs and uses pretrained 2D vision encoders with a cross-attention Stereo Transformer to fuse features and capture 3D structure implicitly. Two variants are presented: StereoPolicy-DP (trained from scratch) and StereoPolicy-VLA (fine-tuning pretrained VLA models). The post details experimental results demonstrating superior performance over RGB-only, RGB-D, and point-cloud-based methods on real-world tabletop tasks and simulations, especially in low-data regimes. It highlights the system's effectiveness on transparent objects and its contribution to physical AI.
  3. One photo in. A full 3D scene out.Introduces PixARMesh, a novel method for single-image 3D scene reconstruction that uses an autoregressive Transformer to tokenize image, object pose, and mesh into a single sequence, enabling direct generation of artist-ready meshes in one forward pass. Achieves state-of-the-art results on the 3D-FRONT benchmark with improved scene-level F-Score and Chamfer Distance compared to prior methods.