TechPrufer
Get started
Lambda
Lambda logoLambda · Feature trail

3D Object-Centric Scene Representation Learning

3 posts · Aug 2026 – Sep 2026

What this trail covers

The platform now supports 3D-DLP, a novel self-supervised approach to decompose colored 3D scenes into object-centric particles. This method extends Deep Latent Particles (DLPs) to 3D, enabling the prediction of object position, size, and appearance without relying on annotations or pre-trained segmentation models. It addresses challenges in 3D scene modeling, such as the visual binding problem caused by tokenization, by learning representations of objects directly in the latent space. The framework now also incorporates StereoPolicy, which enables robot policies to learn 3D spatial understanding directly from stereo images by fusing features from pretrained 2D vision encoders through a cross-attention Stereo Transformer, capturing 3D structure implicitly without explicit depth map or point cloud estimation. This approach has shown improved success rates on real-world tabletop tasks, particularly with transparent objects and in low-data regimes.

All Lambda trails
  1. Giving robots 3D vision without depth sensorsThis post introduces StereoPolicy, a novel approach for robots to gain 3D perception from stereo images without explicit depth sensors. It replaces single-camera observations with synchronized stereo image pairs, processing them with pretrained 2D vision encoders and fusing features via a cross-attention Stereo Transformer. This method implicitly captures 3D structure by learning spatial correspondence and disparity. The post details two implementations: StereoPolicy-DP (trained from scratch with diffusion policies) and StereoPolicy-VLA (fine-tuning on vision-language-action models). It presents quantitative results showing StereoPolicy's superior success rates on real-world tasks compared to RGB-only, RGB-D, and point-cloud-based methods, especially for transparent objects and in low-data scenarios. The work highlights the broader trend towards physical AI and the increasing demand for GPU compute for training and deployment.
  2. How SPREEAI trains the model behind photorealistic virtual try-onThis post details SPREEAI's approach to training a unified diffusion model for photorealistic virtual try-on, addressing pose transfer, cloth deformation, identity preservation, and texture/logo fidelity. It describes a multi-stage training pipeline with rigorous validation gates, high-resolution training, and quantized inference using FP8. The infrastructure leverages NVIDIA H100 GPUs with InfiniBand for efficient gradient synchronization and a rebuilt storage system to manage the large volume of training artifacts. Inference optimizations include targeted upscaling for critical regions, reducing latency and peak memory usage.
  3. From tokens to concepts: how particle models perceive the worldThis post introduces the extension of Deep Latent Particles (DLPs) to 3D scenes, enabling self-supervised decomposition of colored 3D observations into object-centric particles. It details the replacement of the keypoint-proposal mechanism with an appearance-aware K-means prior for clustering in a joint color-and-position space and the implementation of a chroma loss to address color reconstruction issues. The work demonstrates that these 3D particles can drive diffusion-based manipulation policies, outperforming baselines that ignore 3D geometry or use dense voxel inputs.