Overview of NVIDIA Merlin for Recommendation Systems

The presentation introduces NVIDIA Merlin, a comprehensive toolkit for building and deploying recommendation system models. It begins with a product overview, outlining the end‑to‑end workflow that includes feature engineering (NVTABULAR), model training (HugeCTR, Merlin Data Loader), and deployment tools such as the Hierarchical Parameter Server (HPS).

Merlin Models & Systems provide high‑level Python APIs that wrap classic recommendation models (e.g., DLRM, DCN, YouTube DNN) and simplify feature engineering, training, and inference. Users can instantiate models with a single function call and switch between CPU and GPU execution with minimal code changes.

The TensorFlow Distributed Embedding (TFDE) plugin accelerates embedding lookups during training by moving both dense and sparse computations to GPUs, achieving speedups of up to 600× compared to pure‑CPU implementations, and works for both large and small models.

At the lowest level, Merlin Hierarchical‑KV (HKV) is a C++ library that offers a hierarchical, GPU‑backed key‑value store with eviction policies (LRU, LFU, custom). It provides unified memory across CPU/GPU, high performance, and easy integration into existing frameworks.

For inference, the Hierarchical Parameter Server (HPS) caches hot embeddings on each GPU, falling back to CPU memory, SSD, or external backends (e.g., RocksDB) when needed. HPS integrates with Triton, TensorFlow, and PyTorch plugins, delivering low‑latency serving and supporting continuous training pipelines.

Performance benchmarks on DLRM models demonstrate that GPU‑accelerated dense and embedding paths significantly reduce latency across batch sizes, and the system’s modular plugins make it straightforward to adopt in production environments.