1 views collected around this technical thread.
This article explores DeepSeek's open‑source large‑model breakthroughs, PingCAP's AI‑enhanced database roadmap, TiDB.AI's retrieval‑augmented generation framework, the unified TiDB data engine, and practical Q&A insights on knowledge‑graph construction, vector search, and AI‑driven SQL generation.
The paper introduces SOGAIC, a scalable overload‑aware graph‑based index construction system for billion‑scale vector similarity search that uses adaptive overlapping partitioning and load‑balanced distributed scheduling to cut construction time by 47.3% while maintaining high recall.
Redis 8.0, released on May 1 2025, introduces a major license shift to AGPL‑v3, adds eight native data structures—including vector sets, JSON, and time‑series—enhances the query engine with up to 16× performance gains, improves scalability, security, and cloud‑native support, and provides extensive code examples for AI and real‑time analytics.
This article explains the fundamentals of AI agent memory—including short‑term, long‑term, and working memory types and their storage designs—and then details Dify's knowledge‑base segmentation modes, indexing strategies, and retrieval configurations for effective RAG applications.
The article explains how Redis vector search, powered by RedisSearch’s FLAT and HNSW algorithms and supporting various data types and precisions, enables fast AI-driven similarity queries for text, image, and audio, and provides implementation guidance, optimization tips, and a real‑world customer‑service use case.
The article explains how Elasticsearch 8.16.1 stores dense and sparse vectors using various file extensions, compares flat and HNSW index formats, shows how disabling doc‑values removes redundant column‑store copies, and demonstrates scalar and binary quantization—including a quantization‑only mode—that can cut storage to roughly 9 percent while preserving search accuracy.
This guide walks through building a Java RAG system with DeepSeek R1, Milvus, and Spring, covering environment setup, vector model integration via OpenAI protocol, Maven dependencies, data embedding, and a chat endpoint that combines semantic retrieval with LLM generation.
Redis has unveiled a multi‑threaded query engine that dramatically increases query throughput and lowers latency for vector similarity searches, offering up to 16× performance gains and enabling real‑time Retrieval‑Augmented Generation (RAG) workloads in generative AI applications.
Bilibili’s large‑scale recall system separates online processing into a two‑tier merge service and an index service, supports multi‑channel text, item‑to‑item and vector indexes with real‑time updates, uses horizontal sharding, robust CI/CD, monitoring and degradation mechanisms, and is being extended toward model‑based recall and greater automation.
The article explains how Retrieval‑Augmented Generation (RAG) outperforms direct LLM inference by enabling real‑time knowledge updates and lower costs, and demonstrates a practical multi‑modal RAG pipeline that uses Chinese‑CLIP for vector encoding, various chunking strategies, and Redis Search for fast vector storage and retrieval.
This article explores how modern data technologies—from relational databases and NoSQL to vector databases and AI‑driven retrieval—address the 4V challenges of volume, velocity, variety, and value, enabling polyglot persistence, semantic embeddings, and retrieval‑augmented generation for next‑generation applications.
The article traces the progression from the human DIKW information hierarchy to its computer‑world counterpart, illustrates how a homestay platform’s data architecture evolves through relational, NoSQL, search, and data‑warehouse layers, and introduces the next‑generation distributed Data Warebase that unifies structured, semi‑structured, and vectorized knowledge to meet modern AI‑driven business demands.
This article surveys the evolution of modern data technologies—from the 4V characteristics of big data and the limitations of traditional relational databases, through the rise of NoSQL and polyglot persistence, to embedding‑driven vector search, hybrid retrieval and RAG, illustrating how each paradigm frees applications from data constraints.
Redis has launched an enhanced, multi‑threaded query engine that dramatically increases throughput and reduces latency for vector similarity searches, enabling vertical scaling and better support for real‑time RAG applications while maintaining sub‑10 ms response times.
This guide walks entry‑level developers through building a logistics‑focused QA bot by first embedding documents for vector similarity search, then adding retrieval‑augmented generation, fine‑tuning a small model, integrating hybrid checks, and optimizing deployment with feedback loops to achieve fast, accurate, out‑of‑scope‑aware answers.
The article describes a multimodal AI video moderation system built on Alibaba’s Chinese‑CLIP model and hybrid RedisSearch/ElasticSearch vector databases, enabling real‑time violation detection and historical recall, with fine‑tuned black‑market ad detection, FP16 quantization, and OpenVINO acceleration to boost speed and cut storage.
This article explains how Elasticsearch’s advanced search capabilities—including vector and semantic search, hardware acceleration, hybrid retrieval, model re‑ranking, multi‑vector support, and integrated security—enable robust RAG implementations and outlines future directions such as a new compute engine, stronger vector engines, and cloud‑native serverless deployment.
This article explores the evolution of data lakes for AI, discusses the challenges of AI-era data management, introduces Apache Iceberg and its architecture, demonstrates PyIceberg-based AI training and inference pipelines, and presents vector table designs with LSH indexing and performance optimizations.
Redis has launched a multi‑threaded query engine that vertically scales its in‑memory database, dramatically increasing query throughput and lowering latency for vector similarity searches, thereby addressing the performance demands of real‑time retrieval‑augmented generation in generative AI applications.
This article describes how to combine Elasticsearch’s keyword matching with Milvus’s vector‑based semantic search to build a scalable document search service, covering data preprocessing, architecture, query handling, custom scoring, DSL configuration, and result merging.