Free angle observation (Left: RGB, Right: Euclidean Depth).
3D panorama synthesis is a promising yet challenging task that demands high-quality and diverse visual appearance and geometry of the generated omnidirectional content. Existing methods leverage rich image priors from pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic data, but the incompatibility between 3D panoramas and 2D single views limits their effectiveness. In this work, we demonstrate that by applying Multi-plane Synchronization to the operators from 2D foundation models, their capabilities can be seamlessly extended to the omnidirectional domain. Based on this design, we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D panorama generation, which maximizes the reuse of 2D foundation model priors to achieve diverse appearances and accurate geometry while maintaining multi-view consistency. Extensive experiments demonstrate the effectiveness of our approach in panoramic image generation, panoramic depth estimation, and 3D scene generation.
Multi-plane Synchronization synchronizes different 2D spatial operators (attentions, 2d convs, group norms, etc.) to multi-plane panoramic representation, enabling seam-continuous cubemap processing.
The proposed Multi-plane Synchronization is general and can be applied to various 2D diffusion models (e.g., SDXL, Marigold, VideoCrafter2) for both panorama generation and preception tasks:
Based on Multi-plane Synchronization, we further introduce DreamCube, a diffusion-based framework for RGB-D cubemap generation from single-view RGB-D inputs and multi-view text prompts.
DreamCube can generate RGB-D cubemaps from both in-domain and synthetic out-domain inputs.
We provide an interactive demo for free-view observation of the generated RGB-D panoramas. Try using the mouse to drag and rotate the view. Please note that we also provide a Gradio demo which further integrates Pano-to-3D scene reconstruction and free exploration.
Free angle observation (Left: RGB, Right: Euclidean Depth).
Panoramic overview (Left: RGB, Right: Euclidean Depth).
@article{huang2025dreamcube,
title={{DreamCube: 3D Panorama Generation via Multi-plane Synchronization}},
author={Huang, Yukun and Zhou, Yanning and Wang, Jianan and Huang, Kaiyi and Liu, Xihui},
year={2025},
eprint={arXiv preprint arXiv:2506.17206},
archivePrefix={arXiv},
primaryClass={cs.CV},
}