---
title: "Vector Search architecture"
slug: "dev-vector-search-concepts-architecture"
description: "Learn how vector-indexed tables map onto the VAST cluster, enhancing search speed and recall with efficient indexing and querying mechanisms."
updated: 2026-10-30T18:18:51Z
published: 2026-10-30T18:18:51Z
canonical: "docs.vastdata.com/dev-vector-search-concepts-architecture"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Vector Search architecture

*Applies to VAST Cluster 5.5.*

This page explains how a vector-indexed table maps onto the VAST cluster: where the index lives, how it's built in the background, and what happens when you search. It describes the mechanism, so you can reason about speed and recall. For what you choose when you create an index, see [the vector index](/develop/docs/dev-vector-search-concepts-index).

## Design goals

| Goal | What it means | How the index meets it |
| --- | --- | --- |
| Scale to trillions of vectors | The index can't depend on fitting in memory. | The index is a hierarchy of clusters of quantized vectors, kept on the cluster's flash storage. A search reads only the clusters it opens. |
| Fast searches | Search latency close to the time a model takes to run. | Each level of the hierarchy rules out most of the space, and searches run on the CNodes behind the address you connect to. |
| New vectors searchable at once | Data keeps arriving. | New rows can be searched as soon as they commit. Indexing runs in the background and doesn't hold up inserts. |
| Permissions that match the data | Access to vectors can't drift from access to what they describe. | Searches run inside VAST DataBase, so identity policies and row and column security apply to every result. |

## How it maps onto the cluster

![Your application's ADBC client connects to the Query Engine's virtual IP pool, which is dedicated to the Query Engine. The Query Engine runs inside the cluster's CNodes, and the CNodes hold no data of their own. The table's rows, the index's internal structures (the cluster levels and the quantized vectors) and the recent rows that aren't indexed yet sit together on the cluster's shared storage, which every CNode reads. Nothing is split into shards owned by particular servers: every CNode reads the same table and index.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-cluster-mapping.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A27Z&se=2026-11-08T09%3A55%3A27Z&sr=c&sp=r&sig=xO0bZlUWduMO6E2GlI%2BHuPkRv0Q7NsRdhoVXlHEDrFQ%3D)

*Every CNode reads the same table and index from shared storage; nothing is sharded.*

A vector-indexed table is stored like any other VAST DataBase table. Next to the table's rows, the cluster keeps internal structures for the index: the cluster levels, the quantized vectors, and the recent rows that aren't indexed yet. You don't see or query these structures directly. For how VAST DataBase stores tables in general, see [how tables are stored](/develop/docs/dev-database-concepts-architecture-element-store-layout).

Searches run in the VAST Query Engine, which runs inside the cluster's CNodes (see [VAST Query Engine](/docs-5-5/docs/vast-query-engine)). Your client opens an ADBC connection to the Query Engine's virtual IP pool, which is dedicated to the Query Engine. Like everything on the cluster, it's built on the [DASE architecture](/develop/docs/dev-database-concepts-architecture-cluster-architecture): the CNodes hold no data of their own and read the same shared storage, so the index and its table aren't split into shards owned by particular servers.

## The hierarchy

The index is a tree of clusters. Each cluster holds up to 1,000 members of the level below, so each level holds about 1,000 times as many vectors as the one below it:

| Level | Holds | Vectors under one cluster |
| --- | --- | --- |
| L1 | up to 1,000 vectors | 1,000 |
| L2 | up to 1,000 L1 clusters | about a million |
| L3 | up to 1,000 L2 clusters | about a billion |
| L4 | up to 1,000 L3 clusters | about a trillion |
| L5 | the root, where every search starts |  |

The index keeps quantized copies of the vectors. The full-precision vectors (`FLOAT` or `FLOAT16`) stay in the table, and a search reads them only at the end, for the candidates it has picked.

Each cluster above L1 has a centre (a centroid), stored quantized. A search compares the query vector with centroids, level by level, and only opens the clusters whose centres are nearest.

New rows don't join the tree straight away. They wait with the other recent rows until the background indexing places them. Until then, every search reads the recent rows directly and computes their distances one by one, alongside its walk of the tree. So recent rows are always found, but a large backlog of them makes every search do more work. After a big load, give indexing time to catch up before you measure search speed. See [ingest](/develop/docs/dev-vector-search-guides-ingest).

## Quantization in the walk

`cluster_quantization` is the encoding the index uses when it builds and stores its clusters, and `search_quantization` the encoding a search uses for distances while it walks them. Both are set when you create the table; [quantization](/develop/docs/dev-vector-search-guides-quantization) has the allowed values and pairs, and how to choose.

## What happens when you search

![A query vector, a distance function and a LIMIT go to the Query Engine. It walks the index's hierarchy of clusters from the root, through levels L4, L3 and L2, to the L1 clusters that hold the vectors. At each level it compares the query vector with the cluster centres and opens the nearest: at least a minimum and at most a maximum number per level (by default 4 to 8 at L4 and L3, 30 to 200 at L2, and 150 to 3,000 at L1), and between the two only while a cluster is less than 1.9 times as far as the nearest one. Pruned clusters aren't read. Every comparison during the walk uses quantized vectors. From the opened L1 clusters it keeps more candidates than the LIMIT, reads their full-precision vectors once and computes exact distances, merges the recent rows that aren't indexed yet (read directly), and returns the k nearest with the columns you asked for. WHERE filters apply during the walk, so the k rows all pass them. With a distance function that doesn't match the index's metric, ORDER BY descending, more than one sort key, or OFFSET, the index isn't used and the distance is computed for every row.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vector-search-path-v3.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A27Z&se=2026-11-08T09%3A55%3A27Z&sr=c&sp=r&sig=xO0bZlUWduMO6E2GlI%2BHuPkRv0Q7NsRdhoVXlHEDrFQ%3D)

*A search opens only the nearest clusters at each level, and reads recent rows directly.*

1. Your ADBC statement reaches the Query Engine.
2. The planner checks the query's shape. It uses the index only for `ORDER BY` the distance function that matches the index's metric, ascending, with a `LIMIT` and no `OFFSET`, on a vector-indexed table. Any other shape runs as an ordinary scan.  `EXPLAIN` shows the decision: an index search plans a `VAST_VEC_INDEX_SCAN` step, an ordinary scan only a `VAST_SCAN`. See [rules for a search that uses the index](/develop/docs/dev-vector-search-concepts-index#rules-for-a-search-that-uses-the-index).
3. The search walks the tree level by level, from the root. At each level it compares the query vector with the centres of the clusters below and opens the nearest ones: at least a minimum and at most a maximum number for that level. Between the two, it opens the next-nearest cluster only while that cluster is less than a set ratio as far away as the nearest one. So a query whose nearest clusters are clearly closest opens few, and an ambiguous one opens more. By default the budget is 4 to 8 clusters at L4 and L3, 30 to 200 at L2 and 150 to 3,000 at L1, with a ratio of 1.9. Three session settings change it: `vector_search_min_prob`, `vector_search_max_prob` and `vector_search_pruning_distance_ratio`; see [search session settings](/develop/docs/dev-vector-search-reference-configuration).
4. In the L1 clusters it opens, it compares the query vector with every quantized vector and keeps more candidates than the `LIMIT` asks for. Every comparison during the walk uses quantized vectors, in the table's `search_quantization`.
5. It reads the full-precision vectors of those candidates once, computes their exact distances, and merges in the recent rows that aren't indexed yet, which it reads directly. The `k` nearest are returned with the columns you asked for.
6. SQL filters are applied during the walk, not afterwards, so the `k` rows you get all pass them. If a filter rejects many rows, the search keeps opening L1 clusters until enough rows have passed it to fill the `LIMIT`.  See [filters](/develop/docs/dev-vector-search-guides-filters).
7. The results stream back to your client as Arrow data.

## Background work

Two kinds of background work keep the index in shape:

- **Initial indexing.** Once enough recent rows have built up, a k-means pass turns a batch of them into new L1 clusters of about 1,000 vectors each, and places them in the L2 cluster that's currently open. An L2 cluster closes when it holds about 1,000 L1 clusters, about a million vectors. Higher levels are built the same way, from the centres below them.
- **Reclustering.** Clusters that are poorly organized, or emptied by deletes, are rebuilt. For an L2 cluster, all its vectors are redistributed into new L1 clusters; at higher levels, the centres below are redistributed. Reclustering runs k-means on a sample of the vectors, keeps the new clusters evenly sized, assigns every vector to its nearest new centre, and removes deleted rows.

![Three panels show one L2 cluster, about a million vectors in up to 1,000 L1 clusters. After initial clustering, its L1 clusters overlap and vary in size. Reclustering runs k-means on a sample of the vectors and places the same vectors into new, compact, evenly sized L1 clusters. In the steady state the clusters are compact and balanced, and the table's clustering grade is A+. Searches keep running throughout; only the cluster boundaries move.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-reclustering.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A27Z&se=2026-11-08T09%3A55%3A27Z&sr=c&sp=r&sig=xO0bZlUWduMO6E2GlI%2BHuPkRv0Q7NsRdhoVXlHEDrFQ%3D)

*Reclustering re-places the same vectors into compact, evenly sized clusters.*

Both run on the cluster without any action from you, and searches keep running throughout: only the cluster boundaries move. The cluster shares the work fairly between buckets, so one busy table can't hold up the others. Clustering has its own memory budget on the CNodes, but it uses their CPU, so a large load can slow searches until it's done.

The index is a data structure, not a trained model. Nothing needs retraining when your data changes: clusters that no longer fit are reclustered. The cluster tracks each table's progress as its clustering grade (A+, A, B or C), which can jump straight from C to A+ during initial indexing. If it stays low long after a load, contact VAST Support. See [developer observability](/develop/docs/dev-vector-search-guides-developer-observability) and [performance and sizing](/develop/docs/dev-vector-search-operations-performance).

You can't replicate a vector-indexed table on VAST Cluster 5.5; see [limits](/develop/docs/dev-database-reference-limits#combining-table-structures).

## See also

- [The vector index](/develop/docs/dev-vector-search-concepts-index): objects, lifecycle and search rules.
- [Read path and pruning](/develop/docs/dev-database-concepts-architecture-read-path-and-pruning): how VAST DataBase reads tables in general.
- [Table structures: vector-indexed tables](/develop/docs/dev-database-concepts-architecture-table-structures#vector-indexed-tables).
- [Performance and sizing](/develop/docs/dev-vector-search-operations-performance).