Graph Pretraining Techniques for Molecular Representation and Their Applications in Drug Discovery

Introduction: Graph pretraining is mature in general AI but has unique challenges in the biological domain; leveraging massive unlabeled data for compounds and proteins can improve representation learning.

Why pretraining is needed: Labeled data in drug discovery (ADMET, protein‑target affinity) is scarce and expensive, while unlabeled molecular data exceeds 200M; self‑supervised tasks on this data can learn useful embeddings.

Understanding pretraining: Similar to NLP and vision, pretraining learns general knowledge before fine‑tuning on specific tasks; in biology it corresponds to learning basic chemical knowledge such as atom types and bond angles.

Self‑supervised learning: Described mask‑based tasks, context prediction, and examples from NLP (BERT) and image inpainting.

Graph pretraining for compounds: Overview of existing works (PretrainGNN, GROVER, MPG) and their node‑level and graph‑level tasks, highlighting limitations regarding 3‑D structural information.

Our ChemRL‑GEM model: Uses two graph networks—one for atoms‑bonds and another for bonds‑angles—to capture 3‑D geometry; introduces self‑supervised tasks predicting masked atom attributes, bond lengths, bond angles, and inter‑atomic distances.

Experimental results: Evaluated on 12 MoleculeNet benchmarks, achieving state‑of‑the‑art performance on 11 datasets; ablation shows 3‑D‑aware tasks improve results.

Downstream applications: ADMET prediction (improving accuracy by ~4%), protein‑compound affinity prediction (≈2.7% gain), and top rankings in OGB and KDDC competitions.

PaddleHelix platform: Open‑source AI‑driven bio‑computing library offering tools for feature extraction, model configuration, and fine‑tuning of pretraining models for various tasks.

Conclusion: Summarizes the need for pretraining, reviews existing methods, presents ChemRL‑GEM, demonstrates its downstream impact, and introduces PaddleHelix for practical use.