3D Asset Generation with Relightable Gaussians

This post introduces Luce, a novel 3D representation that unifies geometry and physically based rendering (PBR) materials within a voxelized multimodal Gaussian cloud. Luce utilizes dedicated Gaussian primitives for each PBR modality (albedo, metallic-roughness, surface normals) and compresses this representation into a material-aware latent space using a variational autoencoder. A rectified-flow transformer generates this latent from a single image, enabling the creation of relightable, geometrically accurate, and materially faithful 3D assets with preserved fine details. Luce achieves state-of-the-art performance on single-image-to-3D generation benchmarks.