Panoramic Indoor Layout Estimation with Vision Transformer (PanoViT)

The task of indoor layout (layout estimation) aims to infer a 3‑D room model from a 2‑D image, typically by first extracting wall, ceiling, and floor lines and then reconstructing the geometry; panoramic images provide a wider field of view and richer information, making them attractive for this problem.

Existing methods such as LayoutNet, HorizonNet, HohoNet and Led2‑Net rely mainly on convolutional networks, which struggle with global context and noisy or occluded walls, motivating the use of Transformers that excel at capturing long‑range dependencies.

PanoViT consists of four modules: a CNN backbone that maps the panorama to a feature space, a vision‑Transformer encoder that learns global relationships, a layout prediction head that outputs wall, ceiling and floor lines, and a boundary‑enhancement module together with a 3‑D loss to mitigate distortion and emphasize edge information.

The backbone extracts multi‑scale feature maps, while the Transformer encoder employs patch sampling, patch embedding, and multi‑head attention; a recurrent position embedding randomly shifts the horizontal axis during training to focus on relative positions. The boundary‑enhancement module uses frequency‑domain high‑pass filtering (FFT) to highlight line structures, and the 3‑D loss computes errors directly in 3‑D space.

Experiments on Matterport3D and PanoContext datasets using 2DIoU and 3DIoU metrics show that PanoViT achieves state‑of‑the‑art performance, with visualizations confirming accurate wall detection in complex scenes; ablation studies validate the contributions of the recurrent position embedding, boundary enhancement, and 3‑D loss.

To use the model in ModelScope, users open the ModelScope website, search for "panoramic indoor layout estimation", launch the online notebook, upload a 1024×512 panorama, adjust the image path in the provided example code, and run it to obtain wall‑line predictions.