This article explores the technical challenges and unexpected interactions encountered while tuning Approximate Nearest Neighbor (ANN) indexing for a massive 100 million document retrieval system.
The authors detail how instruction-aware query embeddings corrected significant biases toward short documents and analyze the relationship between graph connectivity, search depth, and latency. They also demonstrate how quantization sets an absolute ceiling on recall that cannot be overcome by index tuning alone.
The authors detail how instruction-aware query embeddings corrected significant biases toward short documents and analyze the relationship between graph connectivity, search depth, and latency. They also demonstrate how quantization sets an absolute ceiling on recall that cannot be overcome by index tuning alone.
This article details the process of building a fast vector search system for a large legal dataset (Australian High Court decisions). It covers choosing embedding providers, performance benchmarks, using USearch and Isaacus embeddings, and the importance of API terms of service. It focuses on achieving speed and scalability while maintaining reasonable accuracy.
This article explains the internal workings of vector databases, highlighting that they don't perform a brute-force search as commonly described. It details algorithms like HNSW, IVF, and PQ, the tradeoffs between recall, speed, and memory, and how different RAG patterns impact vector database usage. It also discusses production challenges like filtering, updates, and sharding.
A simple project demonstrating Retrieval Augmented Generation (RAG) using SQLite, sqlite-vec, and OpenAI. It embeds text files, stores them in a SQLite database, and retrieves relevant documents using vector search. The project features lightweight single-file SQLite databases, vector search capabilities, and OpenAI integration for embeddings and chat responses.
Introducing sqlite-vec, a new SQLite extension for vector search written entirely in C. It's a stable release and can be installed in multiple ways. It runs on various platforms, is fast, and supports quantization techniques for efficient storage and search.
