Welcome to the new DOCS.vastdata.com. We've added new menus and a version selector, plus new persona and resource pages to help customers get started. Looking for a previous article? Use the docs search.

Press CTRL+K to open search

Release notes

Prev Next

Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.

This page lists what VAST Cluster 5.5 supports for VAST Vector Search, what each 5.5 release added, and the known limitations. It's a digest of the VAST Cluster 5.5.0 release notes and their service packs. For changes to VAST DataBase as a whole, see the VAST DataBase release notes.

To find out which release your cluster runs, see find your versions. For which Python SDK, ADBC driver and connector versions go with each release, see version compatibility.

What VAST Cluster 5.5 supports

Capability VAST Cluster 5.5
VECTOR(FLOAT, d) and VECTOR(FLOAT16, d) columns, and the SQL distance functions Yes
Vector index, declared when you create the table (VECTOR CLUSTER BY in SQL, create_table(vector_index=...) in the Python SDK, vector_indexes in the VMS REST API) Yes. See create an index.
Distance metrics l2sq, cosine and ip, for the index and for search Yes
Quantization scalar_8bit (the default, for clustering and search) and none, set separately for clustering and search Yes. Not every pair is allowed; see quantization.
RaBitQ quantization No
Secondary index (SECONDARY INDEX, INCLUDE) No. Filter with WHERE; see filters.
Search through the VAST Query Engine (Query Engine virtual IP pool), with ADBC or the Python SDK Yes. See setup.
Search session settings, passed as ADBC connection options Yes. See search session settings.
Python SDK VectorIndexSpec and vector_search() Yes, with SDK 2.1.0. On ip tables, search with SQL instead; see Python SDK.
Index-based search from the Trino or Spark connector No
Add an index to an existing table, or remove one No
Add or drop columns on a vector-indexed table No
Update the vector column No. Delete the row and insert it again. Updating other columns works.
Vector index on a sorted or partitioned table No
Replication of vector-indexed tables No See limits.

Searches use the index only if they have the shape it needs: one distance, ascending, with LIMIT k and no OFFSET. Other searches run as full scans. See query design. For each limit in detail, see limits.

5.5.0

What you can now do

Vector indexing. VAST DataBase tables can now have a vector index. A table with a vector index arranges its data by the vector distances of one vector column, so similarity searches are faster and more accurate and use less compute.

  • You choose the distance metric when you create the index.
  • With the VAST Query Engine, use CREATE TABLE ... VECTOR CLUSTER BY. See create an index.
  • In the VAST Web UI, your platform team enables the Vector indexing capability when they create a table with a vector column, then chooses the column and the distance function (Managing tables).
  • The VMS REST API /tables/ endpoint takes the index as a vector_indexes array, and an index can only be declared when the table is created: /tables/add_columns/ on an indexed table fails with HTTP 400 adding columns on indexed tables is forbidden. See CI/CD and automation.
  • Newly inserted vectors can be searched straight away. The index catches up with them in the background, which can take some time.

The cluster accepts three distance metrics: l2sq (Euclidean), cosine and ip (inner product).

Known limitations

These are listed in the 5.5.0 release notes and repeated unchanged in SP2, SP3 and SP6.

Vector-indexed tables

  • Once vector indexing is enabled for a table, it can't be disabled.
  • After a vector index is created, its distance function can't be changed.
  • Only one vector column can be indexed per table.
  • The word vector can't be used as the vector column name.
  • Adding a vector index to an existing table isn't supported.
  • Adding or deleting columns in an indexed table isn't supported.
  • Vector indexing can't be used together with sorted tables or partitioned tables.
  • In the release notes' words: "VAST Connectors for Trino, Spark and Dremio do not support vector index-based search."

VAST Query Engine

  • Virtual IP pools with the Query Engine role can't be modified after they're created. To change one, your platform team deletes it and creates a new one, so expect the Query Engine address to change if they do. See setup.

5.5.0-SP2

No new vector search features or fixes. The known limitations are the same as 5.5.0. (5.5.0-SP2 release notes)

5.5.0-SP3

No new vector search features or fixes. The known limitations are the same as 5.5.0. (5.5.0-SP3 release notes)

5.5.0-SP6

No new vector search features or fixes. The known limitations are the same as 5.5.0. (5.5.0-SP6 release notes)

See also

Related articles