--- title: "Measure recall and latency" slug: "dev-vector-search-guides-developer-observability" description: "Learn to measure and optimize vector search performance, including recall, latency, and tuning settings for better results in your application." updated: 2026-10-28T02:14:59Z published: 2026-10-28T02:14:59Z canonical: "docs.vastdata.com/dev-vector-search-guides-developer-observability" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Measure recall and latency *Applies to VAST Cluster 5.5.* This guide covers what you can measure from the application side when vector search is slow or returns the wrong neighbours: where query time goes, how to measure recall, how to tune the probe settings against it, and what to send support. Cluster-wide monitoring is a platform-team task; see [performance and sizing](/develop/docs/dev-vector-search-operations-performance). Run the code blocks in order in one Python session. Below each block is the output you should see. ## Check the index first A latency or recall number means little while the index is still catching up with a load: until the cluster has indexed new rows, searches compare the query with each of them, and they're slower ([5.5.0 release notes](/docs-5-5/docs/vast-cluster-5-5-0-release-notes)). Before you measure, check how far the index has got; [performance](/develop/docs/dev-vector-search-operations-performance#check-the-index-first) shows how. Then check that your search uses the index at all. Run `EXPLAIN` on it: a search that uses the index has a `VAST_VEC_INDEX_SCAN` step in its plan; one that scans the table has only a `VAST_SCAN` ([query design](/develop/docs/dev-vector-search-guides-query-design#queries-that-dont-use-the-index)). For a whole load test, your platform team can read the cluster's TopN counters: a search that compares about as many rows as the table holds isn't using the index ([what your platform team can see](/develop/docs/dev-vector-search-operations-performance#what-your-platform-team-can-see)). ## Set up an example The rest of this guide uses two tables with the same 200 rows: one with a vector index, and one without, to compare against. ``` import os import re import time import adbc_driver_manager.dbapi as dbapi import adbc_driver_vastdb BUCKET = os.environ["VASTDB_BUCKET"] SCHEMA = f'"{BUCKET}/vs_guide_observe"' conn = dbapi.connect( driver=adbc_driver_vastdb.get_driver_path(), db_kwargs={ "vast.db.endpoint": os.environ["QE_ENDPOINT"], "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"], "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"], }, autocommit=True, ) def sql(query): with conn.cursor() as cur: cur.execute(query) return cur.fetchall() def vec(i): return [round(((i * 37 + k * 11) % 101) / 100, 2) for k in range(8)] values = ", ".join(f"({i}, {vec(i)}::FLOAT[8])" for i in range(200)) sql(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}") sql(f"CREATE TABLE IF NOT EXISTS {SCHEMA}.indexed (id BIGINT, embedding VECTOR(FLOAT, 8)) " "VECTOR CLUSTER BY embedding (metric = 'l2sq')") sql(f"CREATE TABLE IF NOT EXISTS {SCHEMA}.exact (id BIGINT, embedding VECTOR(FLOAT, 8))") for name in ("indexed", "exact"): sql(f"INSERT INTO {SCHEMA}.{name} VALUES {values}") print(sql(f"SELECT count(*) FROM {SCHEMA}.indexed"), sql(f"SELECT count(*) FROM {SCHEMA}.exact")) ``` ``` [(200,)] [(200,)] ``` ## Time the query, not the pipeline ![A search request in an application has four parts, timed separately: embed the query with your model; connect, which is often the biggest cost in a short request, so reuse the connection; execute the query (execute_s); and fetch the results (fetch_s). Execute returns as soon as the query is accepted, and the search runs while you fetch, so most of the time shows in fetch_s. To check whether you return too much, compare fetch_s with and without the embedding column and any payload you don't need.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-search-timing-v2.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A20Z&se=2026-11-08T09%3A55%3A20Z&sr=c&sp=r&sig=%2BU9AmzpitjczBpiEpwmYjJnL%2BJ9FFb016FETUQMAgTA%3D) *Time the four parts of a search request separately.* A search request in an application usually has four parts: computing the query embedding, connecting, running the query, and fetching the results. Time them separately, so you know which one to work on. Reuse connections: connecting is often the biggest cost in a short request. ``` def search(table, q, k=10): timings = {} t0 = time.perf_counter() with conn.cursor() as cur: cur.execute(f"SELECT id, array_distance(embedding::FLOAT[8], {q}::FLOAT[8]) AS d " f"FROM {SCHEMA}.{table} ORDER BY d LIMIT {k}") t1 = time.perf_counter() result = cur.fetch_arrow_table() t2 = time.perf_counter() timings["execute_s"] = t1 - t0 timings["fetch_s"] = t2 - t1 return result, timings result, timings = search("indexed", vec(7)) print(result.num_rows, sorted(timings)) ``` ``` 10 ['execute_s', 'fetch_s'] ``` With the VAST ADBC driver, `execute()` returns as soon as the query is accepted, and the search runs while you fetch, so most of the time shows in `fetch_s`, even for a small result. To check whether you're returning too much, compare `fetch_s` with and without the embedding column and any payload you don't need. Before you change any search settings, record the values you start from; see [tune the probes](/develop/docs/dev-vector-search-guides-developer-observability#tune-the-probes). ## Measure recall Recall@K is the share of the true K nearest neighbours that the indexed search returns. Measure it against an exact search over the same rows: - Use the same query vectors, metric, K and ID column for both. - Compare **IDs**, not distances. Floating-point distances can differ in the last digits. - Use a set of query vectors that look like your real queries, not one. - For the exact side, use a table without a vector index (as here), or a `none` / `none` [quantization](/develop/docs/dev-vector-search-guides-quantization) copy. ``` def ids(table, q, k): return {row[0] for row in sql( f"SELECT id, array_distance(embedding::FLOAT[8], {q}::FLOAT[8]) AS d " f"FROM {SCHEMA}.{table} ORDER BY d LIMIT {k}")} K = 10 queries = [vec(i) for i in (3, 50, 120, 199)] recalls = [len(ids("indexed", q, K) & ids("exact", q, K)) / K for q in queries] print(f"Recall@{K}: {sum(recalls) / len(recalls):.2f}") ``` ``` Recall@10: 1.00 ``` On a table this small the search sees every row, so recall is perfect. On real data, run the same comparison on a sample of your table and your queries, and repeat it after each change to quantization or search settings. ## Tune the probes The session settings `vector_search_min_prob`, `vector_search_max_prob` and `vector_search_pruning_distance_ratio` decide how many clusters a search reads at each level of the index. More clusters means better recall and slower searches. [Query design](/develop/docs/dev-vector-search-guides-query-design#tune-the-search-with-session-settings) shows how to set them on a connection, and the [configuration reference](/develop/docs/dev-vector-search-reference-configuration#settings) describes each one. - **Record the values you start from.** The cluster's defaults can't be read with SQL. Ask your platform team for the values on your cluster, and write them down before you change anything, so you can go back. - **Measure recall first,** against an exact baseline, as in the previous section. - **Pin the minimum and the maximum to the same values** for a repeatable trial: the search then reads exactly that many clusters at each level. - **Start with the first value in each list.** It's for the lowest level of the index, and it has the most effect on recall ([configuration](/develop/docs/dev-vector-search-reference-configuration#settings)). - **Change one setting at a time,** and measure recall and latency again after each change. - **Don't ship a very large minimum.** A very large minimum on every level approaches an exact search. That's a way to check recall in a test, not a production setting. ## Compare quantization To see how much quantization costs you in recall, compare the table with a copy that uses `none` / `none`, loaded with the same data. [Quantization](/develop/docs/dev-vector-search-guides-quantization#choose-a-setting) describes the steps. ## Give support the query ID When you report a slow or failing query, include its query ID. After `execute()`, read it from the ADBC statement with the option `vast.db.last_query_id`: ``` with conn.cursor() as cur: cur.execute(f"SELECT count(*) FROM {SCHEMA}.indexed") cur.fetchall() query_id = cur.adbc_statement.get_option("vast.db.last_query_id") print(re.sub(r"0x[0-9a-f]+|\d+", "N", query_id)) ``` ``` QueryId(req_id = N, vip_id = N) ``` The example masks the numbers; log the real value. The option is on the statement (cursor), not the connection. ## Clean up ``` for name in ("indexed", "exact"): sql(f"DROP TABLE IF EXISTS {SCHEMA}.{name}") sql(f"DROP SCHEMA IF EXISTS {SCHEMA}") conn.close() print("cleaned up") ``` ``` cleaned up ``` ## Next steps - [Query design](/develop/docs/dev-vector-search-guides-query-design) - [Performance and sizing](/develop/docs/dev-vector-search-operations-performance), for your platform team. - [Troubleshooting](/develop/docs/dev-vector-search-troubleshooting-index)