Recent Advances in Self‑Supervised Learning for Text Recognition (OCR)

1. Introduction

Supervised training of deep neural networks for OCR is limited by the high cost of collecting and labeling large datasets. Self‑supervised learning (SSL) leverages abundant unlabeled images to learn useful representations, first pre‑training on proxy tasks and then transferring to downstream recognition tasks with limited labeled data.

2. Mainstream Text‑Recognition Model Architecture

Modern OCR pipelines treat recognition as a sequence‑to‑sequence problem and typically consist of (optional) geometric transformation, a CNN‑based feature extractor, a sequence‑modeling module (BiLSTM or Transformer encoder), and a decoder (CTC, attention, or Transformer decoder). Recent trends replace the CNN+BiLSTM encoder with a Vision Transformer (ViT).

3. Factors for SSL on Text Images

When designing SSL tasks for text images, researchers must consider the sequential nature of text lines, the uniform style within a line, and whether to include the sequence‑modeling component in the encoder. Unlike generic image SSL, evaluation on OCR uses decoder‑based recognition rather than classification, and downstream tasks may also include text‑image segmentation, super‑resolution, or font manipulation.

4. Representative Papers

4.1 SeqCLR

SeqCLR adapts contrastive learning to text images by splitting feature maps into sequential instances, ensuring that augmentations preserve the order of characters. It uses random image augmentations (contrast, blur, sharpening, small crops, perspective transforms) and an InfoNCE loss. Experiments on handwritten and scene‑text datasets show that SeqCLR outperforms non‑sequential contrastive methods (e.g., SimCLR) in both linear‑probe and semi‑supervised fine‑tuning settings, especially with the “window‑to‑instance” mapping strategy.

4.2 SimAN

SimAN exploits the consistent style of characters within a text line. It crops two adjacent patches, applies only style‑preserving augmentations, and reconstructs one patch from the other using an encoder‑decoder (ResNet‑29 + FCN) with a style‑alignment module based on instance normalization and scaled dot‑product attention. The method combines an adversarial loss with an L2 reconstruction loss. Experiments demonstrate superior probe and semi‑supervised performance compared with SeqCLR, and the learned features transfer well to tasks such as text‑image synthesis and font interpolation.

4.3 DiG

DiG integrates contrastive learning (MoCo‑v3‑style) and masked image modeling (SimMIM) in a unified framework. Two branches process a masked image and an augmented image through a shared ViT encoder. The contrastive branch uses patch‑wise instance mapping and a three‑layer projection head, while the MIM branch masks 60% of patches and predicts pixel values with a linear head. The total loss is a weighted sum of InfoNCE and L2 reconstruction losses. DiG achieves state‑of‑the‑art accuracy on multiple scene‑text and handwritten benchmarks, surpasses prior SSL methods, and also improves downstream tasks like text‑image segmentation and super‑resolution.

5. Conclusion

Self‑supervised learning has shown strong potential for OCR, offering robust feature representations that reduce reliance on synthetic data and improve performance across various text‑image tasks. The article summarizes recent representative works and suggests that continued exploration of SSL will be a key focus for the research team.

References

MoCo, SimCLR, BYOL, MAE, SimMIM and the cited papers (SeqCLR, SimAN, DiG) are listed with their arXiv URLs.