Tag

large data import

1 views collected around this technical thread.

Selected Java Interview Questions
Selected Java Interview Questions
Apr 8, 2024 · Databases

Designing High‑Performance Import of 1 Billion Records into MySQL

This article analyzes how to import one billion 1 KB log records stored in HDFS or S3 into MySQL efficiently, covering constraints, single‑table limits, batch insertion, storage‑engine choices, sharding, file‑reading techniques, task reliability, and concurrency control to achieve optimal throughput.

MySQLPerformance Tuningbatch insert
0 likes · 19 min read
Designing High‑Performance Import of 1 Billion Records into MySQL