Interview with Jianchen: Journey from Open Source Contributor to Data Engineer at Xiaohongshu

This article features an in-depth interview with Jianchen, a data engineer at Xiaohongshu, discussing his journey from a computer science student to a professional in the field of big data storage and message engines. The interview covers his experiences with open source projects, his participation in OSPP (Open Source Summer Program), and his career development.

Jianchen shares his early exposure to open source through courses like MIT6.824 and his initial involvement with SOFAJRaft and Apache RocketMQ communities. He describes how his understanding of open source evolved through three phases: as a learning resource, as a collaborative responsibility, and as an innovative ecosystem.

The interview highlights his two OSPP experiences - in 2021 working on Apache RocketMQ's Dledger-based NameServer for broker automation, and in 2022 building a next-generation storage engine for SOFAJRaft. These projects provided him with valuable engineering and system optimization experience.

Jianchen discusses his decision to join Xiaohongshu after receiving multiple top-tier offers, citing the company's career development support and rapid growth as key factors. He describes his work on Kafka cloud-native transformation, including tiered storage, containerization, and compute-storage separation, which achieved 50% optimization in both storage and compute resources.

The article also covers his transition from student projects to professional work, emphasizing differences in team collaboration, system stability requirements, and business impact. Jianchen offers advice to current students about maintaining curiosity, proactive learning, and effective communication.

His achievements include leading the development of a compute-storage separation Kafka engine (RediantStream) and demonstrating independent project management capabilities within two years of joining Xiaohongshu.