turbovec – Google's Open Vector Index Library

1. What Is turbovec

turbovec is a high-performance vector index library open-sourced by Google Research, implementing the TurboQuant algorithm. Written in Rust with Python bindings, it targets RAG workloads. Data-agnostic quantization needs no training—compress ~10M float32 vectors from ~31GB to ~4GB—and hand-tuned SIMD kernels on ARM and x86 beat FAISS on search speed. Supports search-time filtering, index persistence, and drop-in replacement in major frameworks—efficient for memory-sensitive, low-latency retrieval.

Image source: Official article

Technical positioning and domain: Vector indexing and approximate nearest neighbor search (ANNS) for RAG—high performance, low memory. Data-agnostic quantization without training codebooks distinguishes it from classic trained-quantization indexes.

Research background: Google Research open-sourced TurboQuant to cut training overhead and improve compression vs speed tradeoffs in large-scale indexes. Strong quantization theory and SIMD optimization heritage.

Core value: Fixes FAISS-style k-means training cost, high memory, and inefficient filtered search. Online ingestion: add vectors and search immediately—no rebuild tuning. ~16× compression fits ten-million-scale indexes on modest servers; fully local for privacy.

Technical characteristics: Random orthogonal rotation, TQ+ adaptive calibration, Lloyd-Max scalar quantization, bit-packing—no training, high compression, low distortion. Handwritten NEON and AVX-512BW kernels: ARM 10–19% faster than FAISS IndexPQFastScan; x86 wins at 4-bit. Filter logic short-circuits inside SIMD—no over-fetch, zero recall loss from filtering.

2. Key Features

• Online ingestion index: Vectors searchable immediately after add—no pre-training, tuning, or rebuild on corpus growth—ideal for dynamic datasets.

• Fast SIMD search: Handwritten NEON (ARM) and AVX-512BW (x86)—ARM 10–19% faster than FAISS IndexPQFastScan; x86 4-bit leads. Bit-packing and precomputed buckets minimize latency.

• Search-time filtering: ID allowlists or slot bitmasks filtered inside SIMD kernels—no retrieve-then-filter over-fetch; critical for ACL and hybrid retrieval.

• Stable external IDs and deletion: IdMapIndex supports custom uint64 external IDs and O(1) deletion—stable document IDs for updates.

• Index persistence: write/load to disk without re-encoding—compact format, fast cold start and disaster recovery.

• Framework plug-and-play: Official LangChain, LlamaIndex, Haystack, Agno integrations—swap imports in a few lines.

• Fully local: No hosted service—data stays on machine or VPC for offline RAG stacks.

3. How to Use

1. Requirements: ARM (NEON) or x86 (AVX-512BW); Linux or macOS recommended; Python 3.8+; updated pip.

2. Install: pip install turbovec—prebuilt Rust wheels, no manual C++ build.

3. Create index: TurboQuantIndex(dim=1536, bit_width=4)—2-bit or 4-bit; 4-bit balances compression and accuracy for most cases.

4. Add vectors: index.add(vectors)—auto normalize, rotate, TQ+calibration, Lloyd-Max quantize.

5. Search: index.search(query, k=10)—top-K scores and IDs by inner product; single or batch queries.

6. Persist: index.write("my_index.tv") and TurboQuantIndex.load("my_index.tv").

7. External IDs: Use IdMapIndex for add_with_ids and O(1) delete on stable doc IDs.

8. Hybrid retrieval: SQL/BM25/ACL narrows candidates → pass allowlist for dense rerank.

4. Pros and Cons

5. Comparison with Similar Tools

For memory-tight dynamic RAG—especially ARM edge—turbovec’s online ingest and compression stand out. Max recall on modest data: HNSWlib but heavy RAM and weak native filtering. Max trained precision: ScaNN/FAISS PQ with training cost. turbovec wins on no-training, low latency, and filter-in-kernel for modern RAG pipelines.

6. Editor's Take

turbovec brings Google Research TurboQuant to production: data-agnostic quantization vs FAISS k-means—no training, online ingest, no tuning pain. ARM SIMD gains (10–19% over FAISS) enable edge/mobile RAG. 16× compression puts 10M vectors on ordinary servers; in-kernel filtering boosts hybrid search; local-only satisfies compliance.

Audience: small teams, RAG builders, privacy-sensitive sectors, edge developers. As community and integrations grow, turbovec could become a default RAG index.

Rating: ★★★★☆ (4.5/5) −0.5 for sparse docs, 2-bit x86 competitiveness, smaller community vs FAISS.

7. Use Cases

• Memory-constrained RAG: Index tens of millions of docs on limited RAM—~4GB for 10M vectors with online updates.

• Low-latency online services: Production RAG/recommend/search needing minimal p99—SIMD wins on ARM and x86 4-bit.

• Privacy-first architecture: Government/finance—fully offline RAG, no third-party vector SaaS.

• Edge and mobile: Rust core + ARM optimization on phones, IoT, embedded retrieval.

• Hybrid retrieval: BM25/SQL/ACL prefilter → allowlist dense rerank without over-fetch.

8. FAQ

Q: Training required?
A: No. Quantization is data-agnostic: after add, vectors are searchable with no k-means step and no corpus rebuild. That matters for RAG corpora that grow every day, where FAISS PQ would otherwise need a training pass.

Q: Supported hardware?
A: ARM NEON and x86 AVX-512BW. ARM 10–19% faster than FAISS FastScan; 4-bit x86 leads; 2-bit x86 closer to parity.

Q: LangChain integration?
A: Replace imports with from turbovec.langchain import TurboQuantVectorStore and pass your index instance.

Q: Compression ratio?
A: About 16× at 2-bit (1536-d float32: 6144 B → 384 B) and 8× at 4-bit. Prefer 4-bit for recall/speed; use 2-bit only when RAM is tight and you are on ARM. On x86, 2-bit is closer to FAISS FastScan than the ARM gap suggests.

Q: Delete vectors?
A: Yes. IdMapIndex maps your uint64 document IDs and deletes in O(1)—useful for expiry and ACL revocation. You do not rebuild the whole index; write/load keeps the file searchable after deletes.

Q: vs FAISS advantages?
A: No training, online ingest, faster ARM search, in-kernel filtering, local-only, official framework hooks. FAISS still stronger ecosystem and some low-D recall cases.

9. Project Links

• PyPI: https://pypi.org/project/turbovec/
• GitHub: https://github.com/RyanCodrai/turbovec