--- title: "Setup" slug: "dev-vector-search-operations-setup" description: "Learn how to set up VAST Vector Search in the Query Engine, including permissions, endpoints, and troubleshooting for optimal database performance." updated: 2026-11-05T15:48:01Z published: 2026-11-05T15:48:01Z canonical: "docs.vastdata.com/dev-vector-search-operations-setup" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Setup *Applies to VAST Cluster 5.5.* VAST Vector Search runs in the VAST Query Engine, which is built into the cluster's CNodes. A vector is a column type on VAST DataBase, so you need everything VAST DataBase needs, plus access to the Query Engine. This page lists what to ask your platform team for, gives you a request you can copy, and shows how to check that it all works. ## What you need | You need | What it is | Who provides it | | --- | --- | --- | | A database | A bucket on a Database-enabled view, with a schema for your tables. Exactly as for any VAST DataBase work. | Platform team | | A Query Engine endpoint | The address of a virtual IP pool with the **Query Engine** role, with the cluster's S3 port, for example `http://vastdb-qe.example.com:9090`. The Query Engine needs a pool dedicated to it: until a pool with the Query Engine role exists, Query Engine queries fail on every virtual IP ([errors](/develop/docs/dev-vector-search-troubleshooting-errors#connecting)). | Platform team | | A database endpoint for loading | The ordinary VAST DataBase endpoint, if you load data with the Python SDK (`insert()`, `import_files()`) | Platform team | | An access key pair | An S3 access key and secret key for a VAST user | Platform team | | Permissions | An identity or bucket policy with `s3:QueryEngine*` and `s3:Tabular*`, plus `s3:ListBucket` on the bucket if you use the Python SDK ([below](/develop/docs/dev-vector-search-operations-setup#permissions)) | Platform team | | The ADBC driver | The VAST Query Engine ADBC driver on your client (Linux x86_64). In Python: `pip install adbc-driver-vastdb adbc-driver-manager`, or `pip install 'vastdb[adbc]'` for the SDK. | You | There's no separate switch or setting for vector search. You turn it on per table, when you create the table with a vector index (`VECTOR CLUSTER BY` in SQL, `VectorIndexSpec` in the Python SDK, or the **Vector indexing** capability in the VAST Web UI). See [create an index](/develop/docs/dev-vector-search-guides-create-an-index). **Admin task: send to your platform team.** The setup steps are in the KB: - [Creating Virtual IP Pools for the VAST Query Engine](/docs-5-5/docs/vast-query-engine#creating-virtual-ip-pools-for-the-vast-query-engine): a virtual IP pool with the Query Engine role, and an FQDN in DNS for it. - [Accessing VAST DataBases using the Query Engine](/docs-5-5/docs/vast-query-engine#accessing-vast-databases-using-the-query-engine): the user, keys and policy. - [Setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites): the database, the ordinary endpoint and the SDK's permissions. A Query Engine pool can't be modified after it's created. To change it, your platform team deletes it and creates a new one (a known limitation in 5.5). So agree on its size and addresses before it's created. ![Six items, in order, then your application connects. 1, platform team: a database, a bucket on a Database-enabled view, with a schema for your tables. 2, platform team: a virtual IP pool with the Query Engine role, and an FQDN in DNS for it; the pool can't be modified after it's created, and until one exists, Query Engine queries fail. 3, platform team: the ordinary database endpoint, for loading with the Python SDK's insert() and import_files(). 4, platform team: an S3 access key and secret key for a VAST user. 5, platform team: an identity policy with s3:QueryEngine* and s3:Tabular*, plus s3:ListBucket on the bucket for the Python SDK. 6, you: the VAST Query Engine ADBC driver, adbc-driver-vastdb and adbc-driver-manager, on Linux x86_64. Then your application connects through ADBC: vast.db.endpoint is the Query Engine address with the cluster's S3 port, for example http://vastdb-qe.example.com:9090.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-setup-flow.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A03Z&se=2026-11-08T09%3A55%3A03Z&sr=c&sp=r&sig=Lb6k5NlmZ%2BBrsF9JfCGvtjJ2Aqvd7PI2sn7HLi6MWhA%3D) *What your platform team sets up for Vector Search, in order, before your ADBC application connects.* ## Which endpoint does what ![Two lanes: creating and loading, and searching. In the create and load lane, the Python SDK's create_table() with a vector index, insert() and import_files() go over the Tabular API (HTTP with Arrow data) to the database endpoint, any virtual IP of a pool with the database protocol; a SQL client's CREATE TABLE with VECTOR CLUSTER BY and INSERT go through the ADBC driver to the Query Engine endpoint. In the search lane, both the Python SDK's table.vector_search() and a SQL client's SELECT ordered by a distance with a LIMIT go through the ADBC driver to the Query Engine endpoint; the SDK opens that connection itself, on the session's endpoint. The Query Engine endpoint is a virtual IP pool with the Query Engine role. Everything ends in the same VAST DataBase table, with its rows, its index and its recent rows. Trino and Spark read the table through their VAST connectors but don't use the vector index. The identity policy needs s3:Tabular* for the Tabular API, and s3:QueryEngine* as well for the Query Engine.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/vs-endpoints-v4.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A03Z&se=2026-11-08T09%3A55%3A03Z&sr=c&sp=r&sig=Lb6k5NlmZ%2BBrsF9JfCGvtjJ2Aqvd7PI2sn7HLi6MWhA%3D) *Searches go through ADBC to the Query Engine endpoint; SDK loads go over the Tabular API to the database endpoint.* | Work | Endpoint | | --- | --- | | SQL through ADBC: `CREATE TABLE ... VECTOR CLUSTER BY`, `INSERT`, searches | The Query Engine endpoint | | `table.vector_search()` in the Python SDK | The endpoint of the SDK session. The SDK gives the ADBC driver the same endpoint as the session, so open that session on the Query Engine endpoint. | | Bulk loading with the Python SDK (`insert()`, `import_files()`) | The ordinary database endpoint. You can use a separate session for this. | Trino and Spark connect through their VAST connectors, which don't use the vector index, so don't search through them ([limits](/develop/docs/dev-vector-search-reference-limits)). Point the driver's `vast.db.endpoint` option at the Query Engine address, with its port, for example `http://vastdb-qe.example.com:9090`. A URL without the right port fails with `Connection refused (os error 111)`. See [ADBC](/develop/docs/dev-vector-search-integrations-adbc) and [Python SDK](/develop/docs/dev-vector-search-integrations-python-sdk) for the connection details. ## Environment variables The Vector Search pages read your connection details from these environment variables: ``` export QE_ENDPOINT="http://vastdb-qe.example.com:9090" # the Query Engine endpoint: ADBC, and SDK sessions that call vector_search() export AWS_S3_ENDPOINT_URL="http://vastdb.example.com:9090" # the database endpoint: SDK sessions that only create or load export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export VASTDB_BUCKET="" ``` `vastdb.connect()` reads the three `AWS_*` variables itself. An SDK session that calls `vector_search()` connects with `endpoint=os.environ["QE_ENDPOINT"]`. `QE_ENDPOINT` and `VASTDB_BUCKET` are only a convention of these pages. ## Permissions Searching and DDL through the Query Engine need two groups of actions: - **`s3:QueryEngine*`** for the Query Engine itself. The wildcard covers the Query Engine's individual actions, such as `s3:QueryEngineStartQuery` and `s3:QueryEngineGetData`. Ask for the wildcard. - **`s3:Tabular*`** for DDL and DML on the database. If you also use the Python SDK, its first call on a database needs **`s3:ListBucket`** on the bucket ([setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites#permissions)). **The policy must be attached to your user in your tenant.** An identity policy belongs to a tenant, and a user's policies are attached tenant by tenant. If your database is in a tenant other than the default and your user comes from the cluster's default local provider, attaching the policy without naming the tenant attaches it in the default tenant, where it doesn't apply to your work. To attach it in your tenant, your platform team names the tenant: `user modify --id <user id> --tenant-id <tenant id> --identity-policies-ids <policy ids>` in the VAST CLI, or `PATCH /api/users/<user id>/tenant_data/` with `tenant_id` and `s3_policies_ids` in the REST API. `GET /api/users/<user id>/tenant_data/?tenant_id=<tenant id>` shows the policies that apply in that tenant. If the policy isn't attached in your tenant, every Query Engine request fails with `request was denied according to policies` ([errors](/develop/docs/dev-vector-search-troubleshooting-errors#connecting)), even `SELECT 1`. If your user owns the bucket, the Python SDK's `insert()` and `select()` still work, because they don't need the policy, so the database looks fine until you search. To run queries as an end user ([impersonation](/develop/docs/dev-database-operations-security#impersonation)), the service account's user also needs **`s3:EndUserImpersonation`**, and you pass the end user's name in the `vast.db.end_user` connection option. `s3:Tabular*` doesn't include it. If your platform team uses row and column security, see [security](/develop/docs/dev-vector-search-operations-security). ## What to send your platform team Copy this and fill it in: ``` I need access to VAST Vector Search (VAST DataBase with the VAST Query Engine) for . - Database: an existing database (bucket) , or please create one, with a schema . - Query Engine endpoint: the address or DNS name, and the port, of a virtual IP pool with the Query Engine role, reachable from . - Database endpoint (for bulk loads with the Python SDK): a virtual IP pool address that serves VAST DataBase. - Credentials: an S3 access key pair for a user . - Permissions: an identity policy for that user allowing s3:QueryEngine*, s3:Tabular* on and /*, and s3:ListBucket on , attached to the user in the tenant (user modify --tenant-id, or PATCH /api/users//tenant_data/ with tenant_id). - (If running queries as end users) s3:EndUserImpersonation for the service account. ``` In the VAST Web UI's identity policy builder, the Query Engine actions are listed under **Available Actions** > **Database** > **Query Engine**. ## Check that it works This creates a small vector-indexed table, searches it and drops it. If it prints the two nearest rows, your endpoint, keys, policy and driver are all in place. It uses the [environment variables](/develop/docs/dev-vector-search-operations-setup#environment-variables) above. ``` import os import adbc_driver_manager.dbapi as dbapi import adbc_driver_vastdb conn = dbapi.connect( driver=adbc_driver_vastdb.get_driver_path(), db_kwargs={ "vast.db.endpoint": os.environ["QE_ENDPOINT"], "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"], "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"], }, autocommit=True, # without it, DDL that isn't committed is lost when the connection closes ) cur = conn.cursor() schema = f'"{os.environ["VASTDB_BUCKET"]}/vs_ops"' cur.execute(f"CREATE SCHEMA IF NOT EXISTS {schema}") cur.execute(f""" CREATE TABLE IF NOT EXISTS {schema}.setup_check (id INTEGER, embedding VECTOR(FLOAT, 4)) VECTOR CLUSTER BY embedding (metric = 'l2sq')""") cur.execute(f""" INSERT INTO {schema}.setup_check VALUES (1, [1.0, 1.0, 1.0, 1.0]), (2, [2.0, 2.0, 2.0, 2.0]), (3, [9.0, 9.0, 9.0, 9.0])""") cur.execute(f""" SELECT id, array_distance(embedding::FLOAT[4], [1.0, 1.0, 1.0, 1.2]::FLOAT[4]) AS dist FROM {schema}.setup_check ORDER BY dist LIMIT 2""") for row_id, dist in cur.fetchall(): print(row_id, round(dist, 3)) ``` ``` 1 0.2 2 1.908 ``` If it fails, look up the message in [errors](/develop/docs/dev-vector-search-troubleshooting-errors). A wrong secret key fails on the first statement with `SignatureDoesNotMatch`. Clean up: ``` cur.execute(f"DROP TABLE IF EXISTS {schema}.setup_check") cur.execute(f"DROP SCHEMA IF EXISTS {schema}") cur.close() conn.close() print("dropped") ``` ``` dropped ``` ## Snapshots and replication A vector-indexed table is a VAST DataBase table, so [snapshots and replication](/develop/docs/dev-database-operations-snapshots-and-replication) in the VAST DataBase docs applies to it, with one exception: - **Replication isn't supported.** On VAST Cluster 5.5 a vector-indexed table can't be replicated, and you can't create one on a replicated path. If the database is on a path your platform team replicates, ask before you create indexed tables there. See [limits](/develop/docs/dev-vector-search-reference-limits). You can search a vector-indexed table as it was in a snapshot. Through a snapshot, the SDK's `select()` and `vector_search()` and a Query Engine search all return only the rows the table had when the snapshot was taken. In the Python SDK, get the table from `tx.bucket(BUCKET).snapshot(name)`; in SQL, name the table `"<bucket>/.snapshot/<snapshot name>/<schema>".<table>`. [Snapshots and replication](/develop/docs/dev-database-operations-snapshots-and-replication#find-a-snapshot) shows how to find a snapshot. ## See also - [Security](/develop/docs/dev-vector-search-operations-security): permissions, row and column security, and keeping keys safe. - [Performance and sizing](/develop/docs/dev-vector-search-operations-performance): what to agree with your platform team before a load test. - [Setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites): the VAST DataBase setup this page builds on.