VAST Vector Search cheat sheet

Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.

A one-page summary of VAST Vector Search in VAST Cluster 5.5: create a vector-indexed table, search it, and connect. Each item links to the page with the details.

Create a table with a vector index

CREATE TABLE "bucket/schema".t (
  id BIGINT,
  embedding VECTOR(FLOAT, 1024)
)
VECTOR CLUSTER BY embedding (
  metric = 'l2sq',                      -- l2sq | cosine | ip
  cluster_quantization = 'scalar_8bit', -- default if omitted
  search_quantization = 'scalar_8bit'   -- default if omitted
);

There's no CREATE VECTOR INDEX: the index is created with the table, and can't be added, changed or dropped later. See SQL reference.

Search

SELECT id,
  array_distance(embedding::FLOAT[1024], ARRAY[...]::FLOAT[1024]) AS dist
FROM "bucket/schema".t
ORDER BY dist LIMIT 10;

To use the index: one distance expression in ORDER BY, ascending, with LIMIT k. No OFFSET, and no similarity sorted descending. EXPLAIN shows VAST_VEC_INDEX_SCAN when the index is used. See query design.

Quantization pairs

See quantization.

Connect

• Endpoint: a VAST Query Engine endpoint, on a virtual IP pool with the Query Engine role, with the cluster's S3 port (for example http://<qe-vip>:9090). See setup.
• Permissions: s3:QueryEngine* and the s3:Tabular* actions on the bucket. The Python SDK also needs s3:ListBucket.
• ADBC options: vast.db.endpoint, vast.db.access_key, vast.db.secret_key. Connect with autocommit=True, or commit, or your DDL is lost. See ADBC.
• Search tuning: vast.db.setting.vector_search_min_prob, vector_search_max_prob and vector_search_pruning_distance_ratio (four comma-separated values each), set on the connection with set_options. See search session settings.

Admin task: send to your platform team. The Query Engine VIP pool and the permissions. See setup and security.

Python SDK

from vastdb import VectorIndexSpec

table = schema.create_table("t", columns,
                            vector_index=VectorIndexSpec(column="embedding", distance_metric="l2sq"))
reader = table.vector_search(vec=q, columns=["id"], limit=10)

The result has a vast_pysdk_vector_dist column. Don't use vector_search() on an ip table: it returns the least similar rows first (Python SDK). Signatures: SDK reference.

Operations

• Clustering progress: the REST API grades it A+, A, B or C (clustering_progress_label). C doesn't mean the index is unused. See performance.
• Replication: not supported for vector-indexed tables. See limits.
• Trino and Spark don't use the vector index: search through the VAST Query Engine. See limits.

See also

• SQL reference
• Limits
• Release notes