1 views collected around this technical thread.
This article presents a comprehensive overview of a log clustering system, detailing its background, architecture based on Filebeat, Kafka, Flink, Elasticsearch, and Grafana, and introduces a continuous clustering algorithm using SimHash and Hamming distance for real‑time log governance and anomaly detection.
This article explains the principles behind Simhash, covering the shortcomings of traditional hash functions, the use of cosine similarity, random projection for dimensionality reduction, locality‑sensitive hashing, random hyperplane hashing, implementation steps, query optimization with the pigeonhole principle, and the algorithm's limitations in short‑text scenarios.
This article explains the principles and implementation of Simhash, covering the shortcomings of traditional hash functions, the use of cosine similarity, random projection for dimensionality reduction, locality‑sensitive hashing, and practical optimizations for large‑scale duplicate detection.
This article explains the concept of information fingerprints, compares traditional set‑equality methods, introduces the Simhash algorithm for high‑dimensional text similarity reduction, and demonstrates how partitioned 64‑bit fingerprints enable efficient duplicate detection on massive web data.
This article describes how Zhihu combats large‑scale spam by applying text and behavior clustering techniques—using Jaccard similarity, SimHash fingerprinting, and Spark‑based graph partitioning—to efficiently identify and group similar spammy content and actions.