Welcome to the new DOCS.vastdata.com. We've added new menus and a version selector, plus new persona and resource pages to help customers get started. Looking for a previous article? Use the docs search.

Press CTRL+K to open search

Automation tools

Prev Next

Applies to VAST Cluster 5.5.

Four tools manage VAST objects from code: the VMS REST API, vastpy, the VAST Terraform provider and the VAST admin MCP server. This page compares what each covers for VAST DataBase, VAST Vector Search, VAST Event Broker and VAST DataEngine, how each authenticates, and which API version each calls. It has no examples: each section's CI/CD page shows the tools doing that component's tasks.

Everything here was checked on a lab cluster running VAST Cluster 5.5.0-SP6, with Terraform provider 4.0.2, vastpy 0.3.22 and vast-admin-mcp 0.2.6.

The tools

Tool What it is Where to get it
VMS REST API The cluster's management API, at https://<VMS address>/api/<version>/<object>/. VAST DataEngine has its own API behind the same address, at https://<VMS address>/api/dataengine/<object>. Built into the cluster
vastpy The official Python client for the VMS REST API, plus the vastpy-cli command. It's schema-less: it builds the URL from your calls, so it reaches any endpoint, including the DataEngine API. pip install vastpy; github.com/vast-data/vastpy
Terraform provider vast-data/vastdata Declarative resources for VMS objects, with import and drift correction. Version 4.x targets VAST Cluster 5.5. Terraform Registry; github.com/vast-data/terraform-provider-vastdata
Admin MCP server vast-admin-mcp An MCP server that lets an AI assistant list cluster objects and, in read-write mode, create a few. For interactive work, not pipelines. pip install vast-admin-mcp; github.com/vast-data/vast-admin-mcp

VAST DataEngine also has its own CLI, vastde, which is the primary tool for DataEngine pipelines in CI. See the DataEngine CI/CD page.

Coverage

REST API vastpy Terraform 4.0.2 Admin MCP server 0.2.6
VAST DataBase Covered. Database views, view policies, identity policies, users and S3 keys; schemas, tables and columns Covered, as REST Partial. View, view policy, identity policy, user, S3 key pair, database endpoints (data source). No schemas or tables. S3 keys can't be imported Partial. Lists views, view policies, identity policies and S3 keys; creates a database view in read-write mode
VAST Vector Search Covered, as DataBase, plus vector-indexed tables through /tables/ Covered, as REST None vector-specific. The DataBase resources provision the database; create vector tables with the SDK None vector-specific
VAST Event Broker Covered. Kafka views, topics (including compaction settings), blob expansions Covered, as REST Partial. Kafka view, topics (no compaction, cleanup_policy or max_message_bytes settings), blob expansions (the target table must already exist; there's no table resource) None. Lists which views have KAFKA, but can't create a working Kafka view and has no topic tools
VAST DataEngine Covered through the DataEngine API: registries, functions, triggers, pipelines, deploy, logs and traces Partial. Works with a user name and password; see the limits below None. No function, trigger or pipeline resources None

vastdata_kafka_broker in the Terraform provider registers external Kafka brokers. It doesn't manage VAST Event Broker.

Which tool fits

You want Use
Declared state in Git, with drift detection and correction Terraform, where it has resources for your objects
An idempotent script with no state file, or objects Terraform doesn't have (schemas, tables, compacted topics, DataEngine objects) vastpy, or vastpy-cli in a shell step
A one-off call from a shell, or a runner without Python The REST API with curl
DataEngine functions and pipelines from YAML in Git vastde; the REST API or vastpy when the runner can't have the binary
Looking around the cluster with an AI assistant The admin MCP server, read-only

Authentication

Tool Credentials Tenant TLS
REST API (VMS objects) A VMS API token (Authorization: Api-Token <token>), a user name and password (basic auth), or a JWT (Authorization: Bearer <token>) A tenant admin sends X-Tenant-Name with basic auth Your HTTP client's settings
REST API (DataEngine) A user name and password, or a JWT from POST /api/token/<tenant>. API tokens are rejected (see below) Tenant admin: X-Tenant-Name. Cluster admin: tenant_name as a query parameter Your HTTP client's settings
vastpy token=, or user= and password= (sent as basic auth on every request). CLI: VMS_TOKEN, or VMS_USER and VMS_PASSWORD tenant= (CLI: VMS_TENANT_NAME), sent as X-Tenant-Name Not checked unless you pass a CA file (cert_file=, VMS_CERT_FILE); VMS_CERT_SERVER sets the name to check
Terraform api_token, or username and password (environment variables VASTDATA_API_TOKEN, VASTDATA_CLUSTER_USERNAME, VASTDATA_CLUSTER_PASSWORD) tenant Checked unless skip_ssl_verify = true
Admin MCP server A password for a tenant admin, stored by the OS keyring Chosen at setup

Things to know:

  • Create API tokens with POST /apitokens/ (KB: create API token). The owner must be a manager in the same tenant as the caller. The token value appears only in the response. Store it as a masked CI secret.
  • The DataEngine provisioning API rejects VMS API tokens, from a tenant admin or a cluster admin, with 422 {"detail":"{\"authorization\": \"Value error, Invalid Api-Token\"}"}. Use a user name and password, or get a JWT with POST /api/token/<tenant> (body {"username": ..., "password": ...}; the access field of the response is the token, valid for 1 hour by default). A JWT from POST /api/token/ without the tenant, sent with X-Tenant-Name, is refused with 403 token_not_valid. The DataEngine telemetry endpoints (/api/dataengine/telemetries/logs and /traces) do accept an API token.
  • A cluster admin calling the DataEngine API names the tenant with a tenant_name query parameter, on GET and POST. Without it, the API answers 400 No tenant was provided in request; in the JSON body it's refused with 422 Extra inputs are not permitted; and X-Tenant-Name gets 403 Invalid username/password. vastpy can't add query parameters to a POST, so it can't create DataEngine objects as a cluster admin. Use a tenant admin.
  • vastpy and the DataEngine API: a .post() or .delete() with no arguments sends a request the DataEngine service refuses with 400 Invalid HTTP request received. Call .request("POST") instead, for example for a deploy. The DataEngine CI/CD page has the details.
  • Terraform 4.0.2 needs host in the configuration. VASTDATA_HOST alone fails validation with The argument "host" is required. Pass it as a variable.
  • The admin MCP server can't be set up with a tenant admin's API token: setup reads the clusters endpoint, which a tenant admin isn't allowed to read (403). A password works. On a headless Linux host without a desktop keyring, the password ends up in a plain file under ~/.local/share/python_keyring/.

API versions

VMS serves several versions of its REST API side by side. On VAST Cluster 5.5:

Path What you get
/api/v8/... The 5.5 API
/api/latest/... The same as /api/v8/ (identical objects, apart from their own url)
/api/... (no version) The oldest version's shape. A view has 57 fields instead of 70, without the kafka_* fields and other newer settings, so an unversioned call returns no Event Broker settings.
/api/v9/... 404 {"detail":"Version is unsupported."}

Use /api/v8/ or /api/latest/ in scripts. vastpy, vastpy-cli and the admin MCP server call /api/latest/ by default (vastpy takes version= to change it). The DataEngine API is at /api/dataengine/ and also answers at /api/latest/dataengine/.

The API's getting started page lists the versions per cluster release.

Limits to know

  • Terraform: S3 key pairs (vastdata_user_key) can't be imported. The secret key is stored in state unless you set pgp_public_key, which must be an ASCII-armored key. Creating a view on a path that already exists takes that view over, with no error, and sets it to your configuration: import it first. Topic resources lack compaction settings. There are no DataEngine resources, and data_engine_enabled on a tenant is read-only.
  • vastpy: TLS isn't checked by default. The CLI has no put.
  • Admin MCP server: read-only by default (vast-admin-mcp mcp --read-write enables the create tools). No tools for topics, schemas, tables, keys or DataEngine. list_dataflow_vast with the DATABASE filter returned nothing on our lab cluster, also while a database client was connected.
  • REST API: an unversioned path gets the oldest API shape (see API versions). The DataEngine provisioning API takes no API tokens.

API documentation

  • The KB reference for 5.5: https://kb.vastdata.com/api-5-5/apidocs (VMS and DataEngine APIs). For the DataEngine API, start at getting started and authentication.
  • On your cluster: https://<VMS address>/docs/ ("VAST Management Service (VMS) REST API Documentation"). https://<VMS address>/api/ is a Swagger UI, but on 5.5.0-SP6 it lists only a few paths (tokens, login banner, LEDs).
  • The DataEngine API's OpenAPI specifications on your cluster: https://<VMS address>/api/dataengine/openapi.json (provisioning) and https://<VMS address>/api/dataengine/telemetries/openapi.json (logs, traces and metrics).
  • Tool documentation: the Terraform provider, vastpy and vast-admin-mcp.
Related articles