AutoML for Tabular Data: Research, Techniques, and Applications

The presentation introduces AutoML for tabular data, noting that most public AutoML work focuses on neural architecture search for images, while tabular data modeling remains less explored.

Four main topics are covered: the background of AutoML Tables, automated feature engineering, automated hyper‑parameter optimization, and real‑world case studies.

Automated feature engineering consists of four modules: automatic table joining, feature generation (including unary, binary, group‑by, and high‑order operators), feature selection (using permutation feature importance and field‑wise logistic regression), and feature enhancement (handling NLP, image, audio, and graph data).

Feature selection methods such as PFI and FLR are described, highlighting low‑cost, high‑efficiency strategies for identifying useful features.

The AutoCross system for automatic feature crossing is detailed, including its infrastructure, algorithm collection (beam search, FLR), and workflow implementation, with experimental results showing consistent gains on both public and internal datasets.

Hyper‑parameter optimization techniques presented include Random Coordinate Shrinking (RACOS), Successive Halving (SHA), population‑based training (PBT), and double‑layer optimization, each addressing the trade‑off between exploration and exploitation.

Comparisons with Google Cloud AutoML on ten Kaggle competitions demonstrate that the Fourth Paradigm’s AutoML Tables often achieve higher relative rankings.

Future work focuses on improving efficiency (hardware and algorithmic optimizations) and increasing interactivity, allowing users to intervene in the AutoML loop when desired.