0 views collected around this technical thread.
This article explains PHP’s built-in array_unique() function, detailing its syntax, behavior, and flags, and demonstrates how to remove duplicate values from arrays with practical code examples and real-world use cases such as data cleaning, input validation, and aggregation.
The article explains how space‑efficient probabilistic structures such as BloomFilter and Count‑min Sketch enable large‑scale data deduplication, join pruning, real‑time idempotent filtering, and approximate top‑K analytics by trading modest accuracy loss for dramatically reduced storage and faster computation.
This article explains why duplicate rows appear in MySQL tables, demonstrates how to identify them with SELECT queries, and provides step‑by‑step SQL solutions—including safe deletion of all duplicates or retaining a single record per group—using subqueries and temporary tables for efficient cleanup.
This article explains why duplicate rows appeared in production MySQL tables, demonstrates how to identify them with SELECT queries, and provides two SQL solutions—one to delete all duplicates and another to keep a single record per duplicated key—while preserving data integrity.
This article explains how to identify and remove various kinds of duplicate data in MySQL—including fully duplicated rows, records with duplicate non‑key columns, and unwanted whitespace inside fields—by using SQL statements, table cloning, OS utilities, and regular‑expression updates, with performance measurements for each method.
This article presents a Python script that merges multiple CSV files from a specified directory and removes duplicate rows using pandas, providing a practical solution for test case management.
This article explains why Spark Streaming combined with Kafka can only guarantee at‑least‑once delivery, outlines the challenges of delayed and out‑of‑order events, and presents practical offline‑repair, deduplication, and output‑format techniques—including code examples—to achieve exact‑once semantics in big‑data pipelines.
This article provides a detailed technical analysis of Huawei's Dorado V3 all‑flash storage system, explaining its classification, the GRIP (Granular management, ROW, Inline deduplication & compression, Parity RAID) and FAST (FlashLink, Active‑active, Zero‑loss snapshot, RAID‑TP) technologies, and why these features are essential for modern flash‑oriented solutions.
Recent public SHA-1 collision demonstrated by Google and Dutch researchers highlights the insecurity of SHA-1, prompting a shift toward stronger hashes like SHA-256/3, and underscores the importance of robust hash functions in data deduplication, storage compression, and overall information security.
This article provides an in‑depth, systematic overview of flash storage, covering architecture, metadata management, deduplication, wear‑leveling, power‑loss protection, NAND flash cell types, reliability techniques, emerging 3D‑Flash and memristor technologies, as well as PCIe/NVMe interface standards.
This article explains data deduplication technology, its definition, various classification schemes based on execution time, block size, granularity, and location, and compares it with data compression, highlighting how both techniques can be combined to maximize storage savings.