Vector indexes and metadata filters

I initially understood vector search and metadata filtering as two separate parts of a query.

The vector search ranks documents by similarity. The metadata filter keeps rows that match an exact condition.

WHERE project_id = 'mirror'
  AND created_at >= '2026-01-01'
ORDER BY embedding <=> :query_embedding
LIMIT 5;

That description is correct, but incomplete. The database still has to decide when to apply the filter and how to use the vector index around it.

It may search the global index and filter afterward. It may reduce the candidate set first. It may also combine metadata checks with approximate index traversal. Those plans can have different latency, recall, and result counts even when the SQL looks similar.

What the vector index does

An embedding represents an item as a vector. A distance function such as cosine distance, inner product, or Euclidean distance is then used to rank nearby vectors.

An exact search compares the query with every candidate vector. The terminology varies by system, but this is often called exact KNN or FLAT search.

query vector
    ↓
compare with every eligible vector
    ↓
sort by distance
    ↓
return top K

In Faiss, IndexFlatL2 and IndexFlatIP are exhaustive. In pgvector, nearest-neighbor search is exact by default until an approximate index is added. Exact search is useful as a quality baseline because it returns the nearest neighbors under the selected distance function, apart from ordinary tie and floating-point details.

Approximate nearest-neighbor search, or ANN, checks only part of the search space. It usually reduces query work, but it can miss a true nearest neighbor because that vector was never visited.

The main methods discussed here are:

Faiss documents these index structures directly. ANN is the larger category: HNSW is graph-based, while IVF is partition-based.

HNSW and IVF

HNSW builds a hierarchy of proximity graphs. Search starts in a sparse upper layer, moves toward a promising region, then descends into denser layers.

sparse layer  → broad moves
middle layer  → closer region
dense layer   → nearby candidates

The original HNSW paper describes a multilayer graph with distance scales separated across its hierarchy. At query time, efSearch controls how widely the implementation explores. Larger values generally improve recall and increase query work.

IVF partitions the vector space around learned centroids. Each stored vector is assigned to an inverted list. At query time, the system selects nprobe lists and compares the query with vectors inside them.

query vector
    ↓
find nearby centroids
    ↓
probe selected lists
    ↓
rank the vectors found

Faiss estimates that the fraction scanned is roughly nprobe / nlist, while noting that real lists are uneven. A nearest neighbor can be missed when its list is not selected. Increasing nprobe searches more lists, which generally improves recall and costs more work.

What Flat, SQ, and PQ mean

The names confused me because they combine two separate choices:

HNSW or IVF  → how the index finds candidate vectors
Flat, SQ, PQ → how those vectors are stored and compared

This gives combinations such as HNSWFlat, HNSWSQ, HNSWPQ, IVFFlat, IVFSQ, and IVFPQ. Exact names vary between libraries, but the split is useful.

Flat is slightly overloaded. A standalone Flat index compares the query with every vector and is exact. In IVFFlat, however, Flat only means that vectors inside the selected IVF lists are stored without compression. IVF may still skip other lists, so the overall search remains approximate. HNSWFlat has the same distinction: it stores full vectors, but graph traversal may still miss a neighbor.

Scalar quantization (SQ) compresses each number independently. A float32 dimension normally uses 4 bytes. With SQ8, that dimension is represented by an 8-bit value using 1 byte, so the vector data is about four times smaller. The rounding can slightly change distance calculations.

Product quantization (PQ) compresses more aggressively. It splits a vector into smaller sub-vectors and replaces each part with the ID of a learned codeword. Search can compare these compact codes without reconstructing every original vector, but the estimated distance is less precise.

For a simple scale comparison, consider one million 768-dimensional vectors:

Flat float32  → 768 × 4 bytes  → 3.07 GB
SQ8           → 768 × 1 byte   → 0.77 GB
PQ96x8        → 96 × 1 byte    → 0.096 GB

These figures cover vector codes only. HNSW still needs memory for graph edges, while IVF needs IDs, centroids, and list structures. Metadata and database overhead are also separate. Faiss documents the encoding sizes as 4 × d bytes for Flat, d bytes for SQ8, and M bytes for PQ with M 8-bit subquantizers.

The practical choice is mostly about memory and acceptable recall. Flat is the safest starting point when full vectors fit. SQ8 is useful when a roughly four-times-smaller representation is enough. PQ matters when vector storage is the harder constraint, usually with recall measured against Flat and, when needed, the best candidates reranked using original vectors.

With Flat storage, HNSW and IVF mainly approximate which candidates are visited. SQ and PQ add another approximation by compressing the vectors themselves.

A measured speed–recall example

It is difficult to make a useful claim such as “HNSW is fast” without a dataset, index configuration, hardware, and recall target.

The Faiss SIFT1M benchmark provides one concrete example. It used one million 128-dimensional SIFT vectors and reported the following results with 20 threads:

These are not production latency estimates. They are useful because they show the trade-off inside one controlled setup: searching more graph candidates or more IVF lists improved recall and increased query time. The exact curve will change with the embeddings, hardware, implementation, filter distribution, and index parameters.

What the metadata filter changes

A metadata filter is an exact eligibility rule:

category = 'shoes'
tenant_id = 42
created_at >= 2026-01-01
status = 'active'

The question is where that rule enters the search plan.

Post-filtering

One plan runs ANN over a broader index, then removes rows that fail the filter.

global ANN top 5
 
1. Doc A   wrong project
2. Doc B   allowed
3. Doc C   wrong project
4. Doc D   wrong project
5. Doc E   allowed
 
final result: Doc B, Doc E

The two returned rows satisfy the filter, but the query requested five. An eligible document ranked just outside the global top five was not considered.

This example is intentionally small, but the same issue appears in pgvector’s documentation. With the default HNSW ef_search of 40, a predicate matching 10% of rows produces about four matching rows on average after the initial scan:

40 scanned candidates × 10% selectivity ≈ 4 matching rows

That estimate assumes the filter is distributed roughly across the scanned candidates. Correlation between metadata and vector neighborhoods can make the actual count better or worse.

Pre-filtering

Another plan applies the metadata condition first, produces a smaller set of eligible IDs, then searches that set.

For example, a filter with 0.02% selectivity over one million rows leaves 200 candidates:

1,000,000 rows × 0.0002 = 200 eligible rows

An exact scan over 200 vectors may be reasonable. That is an arithmetic example, not a general threshold. Whether it wins depends on vector dimensions, storage, the filter index, caching, and the database planner.

HNSW adds another complication. If a filtered-search implementation refuses to traverse ineligible nodes, useful paths through the graph can disappear. A node can be ineligible as a result while still being useful for navigation.

So pre-filtering defines the correct logical candidate set, but the physical execution still depends on the database.

Filter-aware search

Some systems combine metadata checks with approximate traversal. They may use indexed metadata, test membership while traversing HNSW, probe more IVF lists, or continue an approximate scan until enough eligible rows are found.

pgvector documents this behavior explicitly. Its iterative scans can continue through HNSW or IVFFlat until the query finds enough filtered rows or reaches a configured scan limit.

There are also index designs built specifically for filtered ANN. The 2024 ACORN paper reports 2–10× higher queries per second than prior methods on earlier benchmarks at 0.9 recall, more than 30× on its newer benchmarks, and more than 1,000× on a 25-million-vector test. The same paper reports construction time up to 11× that of HNSW and index size up to 1.3× larger.

Those figures belong to the paper’s datasets, baselines, and AWS test machine. I would not treat them as a promise for another database. They are evidence that filter-aware graph structure can improve filtered search, while also moving cost into index construction and storage.

Selectivity affects the useful plan

Filter selectivity is the fraction of the collection that remains eligible.

1,000,000 → 900,000 eligible
90% selectivity; an approximate index may still avoid many comparisons
 
1,000,000 → 200 eligible
0.02% selectivity; an exact filtered scan may be competitive

The distribution matters as much as the count. Eligible vectors may be spread evenly, concentrated in a few IVF lists, or separated by ineligible nodes in an HNSW graph.

Repeated boundaries matter too. If most queries are scoped to one tenant or category, a partial index, partition, or separate table may be easier to reason about than one global approximate index. pgvector recommends considering partial indexes for a few filter values and partitioning when there are many values.

I would treat these as planning options rather than rules. The threshold should come from the actual workload.

How to check the result

The most useful proof is an exact baseline on the same data and distance function.

pgvector recommends monitoring recall by comparing approximate results with exact search. One way to force an exact query for a test is to disable index scans inside a transaction:

BEGIN;
SET LOCAL enable_indexscan = off;
 
SELECT id
FROM documents
WHERE project_id = 'mirror'
ORDER BY embedding <=> $1
LIMIT 10;
 
COMMIT;

Run the approximate query separately with the same filter, query vector, and K. Then compare the IDs:

Recall@10 = approximate results also found in exact top 10 / 10

If eight of the approximate results appear in the exact top ten, Recall@10 is 8 / 10 = 0.8.

For filtered search, I would record at least:

A single query is not enough. The comparison should use a representative query set and the same data snapshot. Otherwise a recall number can look precise while measuring very little.

What top K means

For a filtered query with K = 5, I now try to identify which result the system provides:

A. approximate global neighbors, filtered afterward
 
B. approximate neighbors searched within the eligible set
 
C. exact nearest neighbors within the eligible set

All three can appear behind an API described as vector search with metadata filters. They have different guarantees.

I do not think there is one index or filtering strategy that is best for every workload. A safer starting point is to keep an exact baseline, measure recall and latency on representative filters, and inspect the query plan before tuning the embedding model or increasing ANN search parameters.