--- title: "Quantization" slug: "dev-vector-search-guides-quantization" description: "Learn about quantization settings in VAST Cluster 5.5, how to choose them, and their impact on vector comparison accuracy and performance." updated: 2026-10-30T18:19:06Z published: 2026-10-30T18:19:06Z canonical: "docs.vastdata.com/dev-vector-search-guides-quantization" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Quantization *Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.* Quantization stores a compact copy of each vector, with fewer bits per element, so that the cluster can compare vectors faster and with less data. It costs some accuracy. A vector-indexed table has two quantization settings, and you choose them when you create the table. This guide explains what each one does, which combinations VAST Cluster 5.5 accepts, and how to choose. You can't change quantization after the table is created: SQL `ALTER` of the indexed column isn't implemented, and a VMS REST API change is accepted but ignored. To try another setting, create a copy of the table with that setting and load the data into it. Run the code blocks in order in one Python session. Below each block is the output you should see. ## The two settings ![Two settings, both chosen when the table is created and fixed after: cluster_quantization is used to build the index, grouping the vectors into clusters; search_quantization is used while searching, to compare the query vector with the stored vectors in the clusters the search visits. With search_quantization set to scalar_8bit, the default, a search ranks candidates on the 8-bit copies, keeps more candidates than the LIMIT, then re-ranks them at full precision, so the k rows it returns have exact distances; quantization can only make it miss a true neighbour. With none, it compares full-precision vectors throughout. The allowed combinations, as a grid of cluster against search quantization: none with none, accepted; none with scalar_8bit, accepted; scalar_8bit with scalar_8bit, accepted, the default; scalar_8bit with none, rejected. The rule: search_quantization none needs cluster_quantization none. The cluster checks it at CREATE TABLE; through SQL the message is only Invalid Argument.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-quantization.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A08Z&se=2026-11-08T09%3A55%3A08Z&sr=c&sp=r&sig=Nxdnud05l3shp5GnKqfUjPb9jluflDIuU9kIDam1Vqo%3D) *Two create-time settings; 8-bit search re-ranks at full precision, and only one combination is rejected.* | Setting | Used for | | --- | --- | | `cluster_quantization` | Building the index: grouping the vectors into clusters. | | `search_quantization` | Searching: comparing the query vector with the stored vectors in the clusters the search visits. | The two can differ in one direction: full-precision clustering with 8-bit search is allowed, but not the other way round (see [allowed combinations](/develop/docs/dev-vector-search-guides-quantization#allowed-combinations)). With `search_quantization = 'scalar_8bit'`, a search first ranks candidates by their distance on the 8-bit copies. It keeps more candidates than your `LIMIT` asks for, then reads the full-precision vectors of those candidates and ranks them again exactly. So the distances you get back are exact; quantization can only make the search miss a true neighbour. With `'none'`, the search compares full-precision vectors throughout. ## Values | Value | Meaning | | --- | --- | | `'none'` | Full precision: no quantization. | | `'scalar_8bit'` | 8-bit scalar quantization. The default for both settings. | Values are lower case in SQL and in the Python SDK. The VMS REST API takes `NONE` and `SCALAR_8BIT` (or the lower-case values), and it has no default: a REST request must give both settings ([CI/CD and automation](/develop/docs/dev-vector-search-operations-ci-cd#fields-and-the-errors-you-get)). ## Allowed combinations The rule is: **`search_quantization = 'none'` needs `cluster_quantization = 'none'`**. Everything else is accepted. This table is the one place in these docs that lists the combinations; other pages link here. | `cluster_quantization` | `search_quantization` | Result on 5.5 | | --- | --- | --- | | left out | left out | Accepted: `scalar_8bit` / `scalar_8bit` | | `'scalar_8bit'` | `'scalar_8bit'` | Accepted. Same as the default. | | `'none'` | `'scalar_8bit'` | Accepted | | `'none'` | left out | Accepted: `none` / `scalar_8bit` | | `'none'` | `'none'` | Accepted. Highest recall. | | `'scalar_8bit'` | `'none'` | Rejected | | left out | `'none'` | Rejected: the cluster setting defaults to `scalar_8bit` | Here's each case. Connect first: ``` import os import re import adbc_driver_manager.dbapi as dbapi import adbc_driver_vastdb import vastdb BUCKET = os.environ["VASTDB_BUCKET"] SCHEMA_NAME = "vs_guide_quantization" SCHEMA = f'"{BUCKET}/{SCHEMA_NAME}"' conn = dbapi.connect( driver=adbc_driver_vastdb.get_driver_path(), db_kwargs={ "vast.db.endpoint": os.environ["QE_ENDPOINT"], "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"], "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"], }, autocommit=True, ) def sql(query): with conn.cursor() as cur: cur.execute(query) return cur.fetchall() def try_sql(query): try: sql(query) return "OK" except Exception as e: msg = str(e).replace('\\"', '"') m = re.search(r'failed with: (.*?)", request_id', msg, re.S) or re.search(r"internal error: ([^)]*)\)", msg) return m.group(1) if m else msg sql(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}") print("ready") ``` ``` ready ``` Create one table per combination. The SDK then shows what each accepted table ended up with: ``` cases = { "q_default": "", "q_sq_sq": ", cluster_quantization = 'scalar_8bit', search_quantization = 'scalar_8bit'", "q_none_sq": ", cluster_quantization = 'none', search_quantization = 'scalar_8bit'", "q_none": ", cluster_quantization = 'none'", "q_none_none": ", cluster_quantization = 'none', search_quantization = 'none'", "q_sq_none": ", cluster_quantization = 'scalar_8bit', search_quantization = 'none'", "q_search_none": ", search_quantization = 'none'", } for name, props in cases.items(): result = try_sql(f"CREATE TABLE {SCHEMA}.{name} (id BIGINT, embedding VECTOR(FLOAT, 4)) " f"VECTOR CLUSTER BY embedding (metric = 'l2sq'{props})") print(f"{name:14} {result}") session = vastdb.connect() with session.transaction() as tx: schema = tx.bucket(BUCKET).schema(SCHEMA_NAME) for t in sorted(schema.tables(), key=lambda t: t.name): vi = t.vector_index print(f"{t.name:14} cluster={vi.cluster_quantization} search={vi.search_quantization}") ``` ``` q_default OK q_sq_sq OK q_none_sq OK q_none OK q_none_none OK q_sq_none Invalid Argument q_search_none Invalid Argument q_default cluster=scalar_8bit search=scalar_8bit q_none cluster=none search=scalar_8bit q_none_none cluster=none search=none q_none_sq cluster=none search=scalar_8bit q_sq_sq cluster=scalar_8bit search=scalar_8bit ``` The cluster's message for a rejected combination is only `Invalid Argument`. If you get it, check the rule above. The Python SDK and the VMS REST API reject it too; their messages are in [errors](/develop/docs/dev-vector-search-troubleshooting-errors). The Python SDK takes the same values: ``` from vastdb import VectorIndexSpec VectorIndexSpec(column="embedding", distance_metric="l2sq", cluster_quantization="none", search_quantization="none") ``` ## Choose a setting 1. **Start with the default**, `scalar_8bit` for both. It's what you get if you leave the settings out. 2. **If recall is the problem**, first check the embedding model, the metric and how far the index has got (see [performance](/develop/docs/dev-vector-search-operations-performance#check-the-index-first)). Then create a copy of the table with `none` / `none`, load the same data, and compare the results. That gives you the best recall the index can reach on your data, so you know how much quantization costs you. 3. **Use `none` / `none` only when you need it.** It's slower and reads more data, because every comparison uses full-precision vectors. Choose it only if the comparison in step 2 shows a recall gain you need. A `none` cluster quantization also has a much lower maximum dimension than `scalar_8bit`: check your model's dimension against [limits](/develop/docs/dev-vector-search-reference-limits#index-rules). Measure recall with the method in [measure recall and latency](/develop/docs/dev-vector-search-guides-developer-observability#measure-recall). ## In a CREATE TABLE statement ``` -- default: scalar_8bit for both settings VECTOR CLUSTER BY embedding (metric = 'l2sq') -- full-precision clustering, 8-bit search VECTOR CLUSTER BY embedding (metric = 'l2sq', cluster_quantization = 'none', search_quantization = 'scalar_8bit') -- full precision VECTOR CLUSTER BY embedding (metric = 'l2sq', cluster_quantization = 'none', search_quantization = 'none') ``` These are fragments of a `CREATE TABLE` statement; see [create a vector index](/develop/docs/dev-vector-search-guides-create-an-index). ## Clean up ``` for name in cases: sql(f"DROP TABLE IF EXISTS {SCHEMA}.{name}") sql(f"DROP SCHEMA IF EXISTS {SCHEMA}") conn.close() print("cleaned up") ``` ``` cleaned up ``` ## Next steps - [Create a vector index](/develop/docs/dev-vector-search-guides-create-an-index) - [Design a vector table](/develop/docs/dev-vector-search-guides-table-design) - [Query design](/develop/docs/dev-vector-search-guides-query-design) - [Configuration reference](/develop/docs/dev-vector-search-reference-configuration)