LayoutGCN: A Lightweight Graph Convolutional Network for Visually Rich Document Understanding

LayoutGCN is a novel lightweight algorithmic framework designed for Visually Rich Document Understanding (VRDU). It constructs a fully‑connected graph where each node corresponds to a text block, and edges represent pairwise relationships between blocks.

Document Modeling : Text blocks are treated as graph nodes; their four‑corner coordinates provide layout features, and the document image is processed to obtain visual features. A fully‑connected graph is built by linking every pair of nodes.

Model Architecture :

Text Encoding – uses TextCNN with SAME‑padding to generate sequence‑level representations, followed by layer normalization and dropout.

Layout Encoding – normalizes geometric coordinates and maps them through a fully‑connected layer to high‑dimensional layout embeddings.

Image Encoding – employs CSP‑Darknet to extract global image features; node‑level visual features are obtained via a size‑matched RoI pooling mechanism.

Graph Module – concatenates text and layout embeddings as base features and fuses visual features through an attention‑based gating mechanism, then applies multiple Graph Convolutional Network (GCN) layers to propagate information across the graph.

Downstream Tasks supported by LayoutGCN include sequence labeling, node classification, link prediction, and document classification, each implemented by attaching task‑specific heads (e.g., CRF for labeling, fully‑connected layers for classification).

Experiments on public benchmarks (FUNSD, SROIE, CORD, Train‑Ticket, RVL‑CDIP) show that LayoutGCN achieves results comparable to large pre‑trained models while using far fewer parameters and no pre‑training. Detailed analysis highlights strengths on datasets with moderate layout complexity and limitations on highly diverse layouts.

Case Studies demonstrate successful deployment in Ant Group’s real‑world scenarios such as credential parsing, bill detail extraction, and logistics invoice structuring, confirming the model’s practicality across languages and document types.

Conclusion emphasizes that shallow multimodal features combined with graph‑based relational modeling provide an effective, resource‑efficient solution for VRDU, and outlines future work on improving adjacency weight design and extending to vision‑centric tasks.