RAG Vector Indexes: Interview Answer Framework

Once a candidate can explain embeddings, interviewers move to indexing: how do you actually find the nearest neighbors among millions or billions of vectors fast enough to keep RAG latency acceptable? This question tests whether you understand approximate nearest neighbor (ANN) search as an engineering tradeoff, not a black box a vector database vendor solved for you.

Core Concepts

Exact nearest neighbor search — computing distance from a query vector to every vector in the corpus — is O(n) per query and becomes impractical past roughly 100K-1M vectors, depending on latency requirements. Approximate Nearest Neighbor (ANN) algorithms trade a small amount of recall for orders-of-magnitude speedup, and the specific algorithm choice is a first-class production decision.

HNSW is the default choice for most production RAG systems under ~100M vectors: it builds a multi-layer graph where each node connects to its approximate nearest neighbors, allowing greedy graph traversal to find near-optimal matches in logarithmic time. Its downside is memory — the full graph plus vectors typically needs to live in RAM, which becomes expensive past hundreds of millions of vectors.

IVF partitions the vector space into k clusters (via k-means) and only searches the nprobe closest clusters to the query, trading recall for speed based on how many clusters you probe. It’s cheaper to build and update than HNSW but generally has a worse recall-latency curve at the same memory budget.

Product Quantization (PQ) compresses vectors by splitting them into subvectors and quantizing each to a small codebook, cutting memory 10-30x at the cost of some recall — essential once you’re at billion-scale and can’t afford full-precision vectors in memory. IVF-PQ combines both: cluster first, then compress within clusters.

The three managed vector databases interviewers most often ask about differ mainly in operational model, not core algorithm:

• Pinecone: fully managed, serverless option available, HNSW-based, strong at hybrid metadata filtering, no infrastructure to operate.
• Weaviate: open-source with managed cloud option, HNSW-based, built-in hybrid search (BM25 + vector) and GraphQL query interface, strong module ecosystem.
• Qdrant: open-source, Rust-based (notably fast), HNSW-based with strong filtering performance even under heavy metadata constraints, popular for self-hosted deployments needing tight latency control.

📧 Get free interview prep resources — frameworks and real FAANG questions. Download the free kit →

Interview Answer Framework (4-Step)

Step 1 — Establish scale. The correct index choice is almost entirely a function of corpus size. Ask: how many vectors, and what’s the growth trajectory over 12-24 months? A system at 500K vectors and one at 500M vectors need fundamentally different architectures, and a good answer states this before naming a specific database.

Step 2 — State the recall-latency-cost triangle. Explicitly name the tradeoff: you can pick any two of high recall, low latency, and low memory/cost — the third gets compromised. HNSW favors recall and latency at the cost of memory. IVF-PQ favors memory and speed at some recall cost. This framing alone signals strong systems thinking.

Step 3 — Recommend based on constraints, with a fallback. For most RAG systems under 50M vectors with normal cloud budgets, recommend HNSW via a managed provider (Pinecone/Weaviate/Qdrant) as the default, since the recall ceiling (95-99%) is high enough that retrieval rarely becomes the bottleneck versus reranking or generation quality. Only introduce IVF-PQ or disk-based indexes once you’ve stated a specific memory or cost constraint that HNSW can’t meet.

Step 4 — Name the tuning knobs. For HNSW, that’s ef_construction (build-time accuracy) and ef_search (query-time accuracy/speed tradeoff), plus M (graph connectivity, higher M = better recall, more memory). For IVF, that’s nlist (number of clusters) and nprobe (clusters searched per query). Naming these signals hands-on experience, not textbook knowledge.

Common Follow-ups

“How would you evaluate whether your ANN recall is good enough?” Build a small golden set with known ground-truth nearest neighbors (via brute-force flat search on a sample), then measure Recall@k of your ANN index against that ground truth. If ANN recall drops meaningfully below 95% on your golden set, downstream generation quality often degrades in ways that are hard to attribute back to retrieval.

“What happens at billion-scale?” At that scale, full-precision HNSW in RAM becomes cost-prohibitive (a single 1B-vector, 1536-dim, fp32 index is roughly 6TB just for raw vectors). The standard answer: introduce PQ or scalar quantization to compress vectors, consider disk-resident indexes (DiskANN-style), and often shard the index across multiple nodes with a routing layer, accepting a few points of recall loss in exchange for feasibility.

“How do you handle real-time updates to the index?” HNSW graphs support incremental insertion but degrade over time without periodic rebuilds/compaction as deletions accumulate (most implementations use tombstoning rather than true deletion). For high-churn corpora, plan for periodic full re-index jobs rather than assuming pure incremental updates stay performant indefinitely.

Production Considerations

Filtering interacts badly with ANN indexes if not designed for. Pre-filtering (filter first, then search) can devastate recall when filters are selective, since the ANN graph traversal was built assuming the full vector space. Post-filtering (search first, then filter) risks returning too few results if the filter is strict. Production systems (Qdrant and Weaviate especially) implement filtered HNSW variants that integrate the filter into graph traversal — know this exists and ask about it when comparing databases for filter-heavy use cases.

Reindexing cost is often underestimated: any embedding model change requires a full index rebuild, which for HNSW at scale can take hours and requires either a blue-green index swap or accepting a maintenance window. Budget for this in any system design answer involving index migrations.

For a full walkthrough of the RAG interview surface — embeddings, indexing, hybrid search, reranking, and end-to-end system design — see The 0-to-1 AI Engineer Interview Playbook (Amazon: https://www.amazon.com/dp/B0H2CML9XD?tag=sirjohnnymai-20).

FAQ

Q: Is HNSW always the right default for RAG? A: For most systems under roughly 50-100M vectors with normal cloud infrastructure budgets, yes — it gives the best recall-latency tradeoff without requiring lossy compression. Above that scale, or under tight memory budgets, IVF-PQ or disk-based indexes become necessary.

Q: How much recall loss is acceptable in a production RAG system? A: There’s no universal number, but 95%+ Recall@10 against a golden set is a common production bar, since retrieval errors compound with reranking and generation errors downstream. Measure against your own golden set rather than relying on a rule of thumb.

Q: Do I need a dedicated vector database, or can I use pgvector/Postgres extensions? A: For smaller corpora (under a few million vectors) or teams wanting to avoid a new piece of infrastructure, pgvector with IVFFlat or HNSW support is often sufficient. Dedicated vector databases become worth the operational overhead once you need horizontal scaling, advanced filtering performance, or multi-tenant isolation at scale.