Quantization

Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.

Quantization stores a compact copy of each vector, with fewer bits per element, so that the cluster can compare vectors faster and with less data. It costs some accuracy. A vector-indexed table has two quantization settings, and you choose them when you create the table. This guide explains what each one does, which combinations VAST Cluster 5.5 accepts, and how to choose.

You can't change quantization after the table is created: SQL ALTER of the indexed column isn't implemented, and a VMS REST API change is accepted but ignored. To try another setting, create a copy of the table with that setting and load the data into it.

Run the code blocks in order in one Python session. Below each block is the output you should see.

The two settings

Two create-time settings; 8-bit search re-ranks at full precision, and only one combination is rejected.

The two can differ in one direction: full-precision clustering with 8-bit search is allowed, but not the other way round (see allowed combinations).

With search_quantization = 'scalar_8bit', a search first ranks candidates by their distance on the 8-bit copies. It keeps more candidates than your LIMIT asks for, then reads the full-precision vectors of those candidates and ranks them again exactly. So the distances you get back are exact; quantization can only make the search miss a true neighbour. With 'none', the search compares full-precision vectors throughout.

Values

Values are lower case in SQL and in the Python SDK. The VMS REST API takes NONE and SCALAR_8BIT (or the lower-case values), and it has no default: a REST request must give both settings (CI/CD and automation).

Allowed combinations

The rule is: search_quantization = 'none' needs cluster_quantization = 'none'. Everything else is accepted. This table is the one place in these docs that lists the combinations; other pages link here.

Here's each case. Connect first:

import os
import re
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb
import vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA_NAME = "vs_guide_quantization"
SCHEMA = f'"{BUCKET}/{SCHEMA_NAME}"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def sql(query):
    with conn.cursor() as cur:
        cur.execute(query)
        return cur.fetchall()

def try_sql(query):
    try:
        sql(query)
        return "OK"
    except Exception as e:
        msg = str(e).replace('\\"', '"')
        m = re.search(r'failed with: (.*?)", request_id', msg, re.S) or re.search(r"internal error: ([^)]*)\)", msg)
        return m.group(1) if m else msg

sql(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
print("ready")

ready

Create one table per combination. The SDK then shows what each accepted table ended up with:

cases = {
    "q_default": "",
    "q_sq_sq":   ", cluster_quantization = 'scalar_8bit', search_quantization = 'scalar_8bit'",
    "q_none_sq": ", cluster_quantization = 'none', search_quantization = 'scalar_8bit'",
    "q_none":    ", cluster_quantization = 'none'",
    "q_none_none": ", cluster_quantization = 'none', search_quantization = 'none'",
    "q_sq_none": ", cluster_quantization = 'scalar_8bit', search_quantization = 'none'",
    "q_search_none": ", search_quantization = 'none'",
}
for name, props in cases.items():
    result = try_sql(f"CREATE TABLE {SCHEMA}.{name} (id BIGINT, embedding VECTOR(FLOAT, 4)) "
                     f"VECTOR CLUSTER BY embedding (metric = 'l2sq'{props})")
    print(f"{name:14} {result}")

session = vastdb.connect()
with session.transaction() as tx:
    schema = tx.bucket(BUCKET).schema(SCHEMA_NAME)
    for t in sorted(schema.tables(), key=lambda t: t.name):
        vi = t.vector_index
        print(f"{t.name:14} cluster={vi.cluster_quantization} search={vi.search_quantization}")

q_default      OK
q_sq_sq        OK
q_none_sq      OK
q_none         OK
q_none_none    OK
q_sq_none      Invalid Argument
q_search_none  Invalid Argument
q_default      cluster=scalar_8bit search=scalar_8bit
q_none         cluster=none search=scalar_8bit
q_none_none    cluster=none search=none
q_none_sq      cluster=none search=scalar_8bit
q_sq_sq        cluster=scalar_8bit search=scalar_8bit

The cluster's message for a rejected combination is only Invalid Argument. If you get it, check the rule above. The Python SDK and the VMS REST API reject it too; their messages are in errors.

The Python SDK takes the same values:

from vastdb import VectorIndexSpec

VectorIndexSpec(column="embedding", distance_metric="l2sq",
                cluster_quantization="none", search_quantization="none")

Choose a setting

1. Start with the default, scalar_8bit for both. It's what you get if you leave the settings out.
2. If recall is the problem, first check the embedding model, the metric and how far the index has got (see performance). Then create a copy of the table with none / none, load the same data, and compare the results. That gives you the best recall the index can reach on your data, so you know how much quantization costs you.
3. Use none / none only when you need it. It's slower and reads more data, because every comparison uses full-precision vectors. Choose it only if the comparison in step 2 shows a recall gain you need. A none cluster quantization also has a much lower maximum dimension than scalar_8bit: check your model's dimension against limits.

Measure recall with the method in measure recall and latency.

In a CREATE TABLE statement

-- default: scalar_8bit for both settings
VECTOR CLUSTER BY embedding (metric = 'l2sq')

-- full-precision clustering, 8-bit search
VECTOR CLUSTER BY embedding (metric = 'l2sq', cluster_quantization = 'none', search_quantization = 'scalar_8bit')

-- full precision
VECTOR CLUSTER BY embedding (metric = 'l2sq', cluster_quantization = 'none', search_quantization = 'none')

These are fragments of a CREATE TABLE statement; see create a vector index.

Clean up

for name in cases:
    sql(f"DROP TABLE IF EXISTS {SCHEMA}.{name}")
sql(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()
print("cleaned up")

cleaned up

Next steps

• Create a vector index
• Design a vector table
• Query design
• Configuration reference