How Sohu Changyou Migrated 1 PB of Game Data to the Cloud Without Downtime
This article details how Sohu Changyou’s data team, together with Tencent Cloud engineers, planned and executed a seamless migration of over one petabyte of game data to Elastic MapReduce, Elasticsearch Service and Oceanus, achieving zero service impact and dramatically improving analytics performance.
Moving a petabyte‑scale dataset to the cloud is far more complex than a simple copy, as Tencent Cloud big‑data architect Li Shaobo explains.
When Sohu Changyou prepared to launch the new mobile game "Little Raccoon Hundred Generals," its existing IDC‑based big‑data cluster faced mounting pressure from rapidly growing data volume and diverse analytical workloads.
To meet the need for higher timeliness and stability, the team chose Tencent Cloud’s Elastic MapReduce (EMR), Elasticsearch Service (ES) and streaming compute Oceanus for the migration.
Two‑handed preparation involved close collaboration between Sohu Changyou’s BI team and Tencent Cloud architects. They catalogued data, tasks, and workflows, designed migration schemes for historical and real‑time data, performed sample migrations, and implemented a dual‑write strategy to gradually shift workloads during low‑traffic periods.
During the migration, challenges such as Impala OOM errors and mismatched C++/Java environments required rapid patching and joint debugging, even on weekends.
After thorough testing, the PB‑level data was successfully moved in mid‑July with a parallel “dual‑run” approach to verify stability before fully cutting over.
The migration resulted in zero business impact, a six‑fold reduction in BI offline analysis time (from 2 hours to under 20 minutes), a 60% decrease in operational manpower, and the ability to scale the cluster to 500 cores in under two minutes. Query latency remained sub‑second for 99.995% of requests, and alerts were delivered within three seconds.
Beyond gaming, Tencent Cloud’s EMR, ES, Oceanus, ClickHouse, PostgreSQL, WeData, and DLC products are now widely adopted across finance, e‑commerce, travel, and online education.
Tencent Tech
Tencent's official tech account. Delivering quality technical content to serve developers.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.