ZSTD | BestHub

DataFunSummit

Jan 16, 2025 · Big Data

Zhihu Big Data Cost‑Reduction Practices: FinOps, Erasure Coding, ZSTD Compression, Spark Auto‑Tuning, and Remote Shuffle Service

This article details Zhihu's comprehensive cost‑reduction and efficiency‑boosting initiatives for its big‑data platform, covering FinOps‑driven financial operations, hybrid‑cloud architecture, cost allocation models, operational monitoring, and technical optimizations such as erasure coding, ZSTD compression, Spark auto‑tuning, and a remote shuffle service.

Cloud Cost ManagementErasure CodingFinOps

0 likes · 22 min read

Zhihu Big Data Cost‑Reduction Practices: FinOps, Erasure Coding, ZSTD Compression, Spark Auto‑Tuning, and Remote Shuffle Service

JD Tech

Mar 14, 2024 · Databases

JD ElasticSearch Supports ZSTD Compression: Implementation, Performance Evaluation, and Usage Guide

This article explains how JD ElasticSearch has integrated the high‑performance ZSTD compression algorithm, details the motivations behind its adoption, presents benchmark results comparing it with LZ4 and best_compression, and provides step‑by‑step instructions and code snippets for configuring and using the new jd_zstd codec in Elasticsearch.

BigDataJavaLucene

0 likes · 14 min read

JD ElasticSearch Supports ZSTD Compression: Implementation, Performance Evaluation, and Usage Guide

DaTaobao Tech

Dec 8, 2023 · Backend Development

ZSTD Compression and GC Optimization in Java Netty Backend

Switching a Java Netty gateway from GZIP to ZSTD compression using zstd‑jni doubled GC time and introduced heap and Netty off‑heap memory leaks, which were resolved by employing the library’s direct off‑heap API with a NoFinalizer compressor, promptly releasing ByteBufs, avoiding finalize(), and adopting jemalloc to reduce fragmentation.

GCJavaMemoryLeak

0 likes · 19 min read

ZSTD Compression and GC Optimization in Java Netty Backend

Didi Tech

Aug 10, 2023 · Big Data

Implementing ZSTD Compression in Didi's Elasticsearch for High‑Performance Log Ingestion

By integrating ZSTD compression into Didi’s Elasticsearch 7.6, the team cut CPU usage by about 15 %, reduced index storage roughly 30 %, boosted write throughput up to 25 %, and eliminated over 20 servers, demonstrating a faster, more storage‑efficient solution for petabyte‑scale log ingestion.

Big DataLuceneZSTD

0 likes · 10 min read

Implementing ZSTD Compression in Didi's Elasticsearch for High‑Performance Log Ingestion

Laravel Tech Community

Jul 5, 2022 · Fundamentals

Linux Kernel 5.19 RC5 Released with New Features and Upcoming Stable Version

Linux kernel version 5.19 has entered its fifth release candidate (RC5), with Linus Torvalds noting a low patch count focused on driver fixes, architecture updates, and filesystem and networking improvements, and introducing support for LoongArch CPUs, Apple NVMe, Zstd-compressed firmware, and FAT32 creation timestamps, with the stable release slated for late July.

LinuxLoongArchNVMe

0 likes · 2 min read

Linux Kernel 5.19 RC5 Released with New Features and Upcoming Stable Version

Aikesheng Open Source Community

May 20, 2020 · Databases

MySQL Binlog Transaction Compression with ZSTD (MySQL 8.0.20)

The article explains MySQL 8.0.20's binlog transaction compression feature using the ZSTD algorithm, describes its internal implementation, outlines usage steps, presents performance test results, and lists important considerations for deploying compression in production environments.

MySQLReplicationZSTD

0 likes · 6 min read

MySQL Binlog Transaction Compression with ZSTD (MySQL 8.0.20)