PlugNet: A Plug‑in Super‑Resolution Unit for Low‑Quality Text Recognition in Natural Scene OCR

ImageDT (ImageDT) is a leading AI commercial service provider for the global retail and consumer goods industry, founded in November 2016. The company has launched three core products—Woodpecker, Sky Eagle, and Paul—offering intelligent retail management solutions such as store inspection, product monitoring, and market share analysis.

Natural scene OCR aims to recognize text in complex real‑world images, which differs from traditional OCR that processes scanned documents. Traditional OCR follows a three‑step pipeline: text region detection, geometric correction, and character segmentation, using methods like connected component analysis, SVM, and logistic regression. However, these approaches fail when faced with diverse fonts, colors, low‑resolution, blur, and varying lighting conditions typical of retail, automotive, and other scenarios.

With the rise of deep learning, OCR has shifted to data‑driven models. Two main research directions exist: (1) separating text detection and recognition, using detectors such as EAST, CTPN, or SegLink, and recognizers like CRNN, RARE, which combine CNNs, RNNs, attention, or CTC; (2) building end‑to‑end models that jointly detect and recognize text, exemplified by Mask TextSpotter. These methods improve accuracy but increase computational complexity, especially for rotated or curved text.

Super‑resolution (SR) techniques reconstruct high‑frequency details from low‑resolution images, enhancing the quality of low‑quality text images without hardware changes. SR has been applied in fields like remote sensing, medical imaging, and video processing, with popular models ranging from SRCNN and EDSR to GAN‑based SRGAN and ESRGAN. Incorporating SR into OCR can aid low‑quality text recognition, though naive integration may increase memory and energy consumption.

PlugNet, presented at ECCV, introduces a Plug‑in Super‑Resolution Unit (PSU) that can be attached to an OCR backbone to boost low‑quality text recognition. The architecture consists of (1) a text rectification module inspired by STN, (2) a shared convolutional backbone for both OCR and SR branches, (3) the detachable PSU, and (4) the OCR branch with feature sequencing and a bidirectional LSTM with attention. The PSU comprises residual channel attention blocks that enhance high‑frequency features while allowing removal during deployment to reduce inference cost.

Experiments demonstrate that the shared backbone improves baseline OCR performance, and the PSU outperforms ESRGAN‑based SR integration, achieving state‑of‑the‑art results on several benchmark datasets, especially those with a high proportion of low‑quality text such as SVT and SVTP. Multi‑task training balances OCR cross‑entropy loss and SR L1 loss, with a lambda weight of 0.01 yielding optimal performance.

The work concludes by highlighting the need for further research on integrating SR more tightly with OCR for cloud deployments and developing fully end‑to‑end trainable detection‑recognition‑SR models, aiming to further enhance low‑quality text recognition in diverse real‑world scenarios.