Applies to VAST Cluster 5.5.
These docs are for developers, data engineers and data scientists who build on the VAST cluster: write applications, pipelines and queries against its services. They cover six services, each in its own section, and are written for VAST Cluster 5.5.
These docs and the KB
The VAST knowledge base (KB) answers "how do I enable, configure or operate this on the cluster". These docs answer "how do I build with this". Enabling a service, creating views, users, keys and policies, and other admin procedures are tasks for your platform team, so these pages link to the KB for them instead of repeating them. Each service section has a setup or access page that lists what to ask for; get access brings them together.
Links to the KB go to kb.vastdata.com and leave these docs.
The six services
- VAST DataBase is a transactional database for tables, built into the VAST cluster, which you work with from the Python SDK, Trino, Spark or the VAST Query Engine.
- VAST Vector Search is nearest-neighbour search over embeddings stored in VAST DataBase tables: a vector is a column type, and you search with SQL through the VAST Query Engine.
- VAST Event Broker is a Kafka-compatible event broker built into VAST Cluster, where each topic is stored as a VAST DataBase table.
- VAST DataEngine runs your Python functions on Kubernetes when something happens to your data on the cluster, or on a schedule.
- VAST DataStore is the file and object storage of a VAST cluster, which applications reach over S3, NFS and SMB.
- VAST Catalog is a table of metadata about the files, objects and directories on your cluster; the same section covers the audit log, a table of the protocol operations clients performed.
Six services on one cluster, and the clients that reach them.
Not sure which one you need? See choose a service. For which client reaches which service, see choose a client.
How each service section is organised
Every service section has the same parts, in the same order in the navigation. It opens with its own pages, then its folders:
| Part | What it holds |
|---|---|
| Overview and Quickstart | What the service is, what it's for and what to watch for, and a quickstart you can paste and run in about 10 to 15 minutes. |
| Concepts and Architecture | The objects you work with and how they behave, and how the cluster runs them. VAST DataBase has an Architecture folder with several pages. |
| Release notes | What each 5.5 release changed, in the sections that have them. |
| Guides | Task-by-task pages, each with runnable code. |
| Best practices | Recommendations for particular workloads. |
| Integrations | Clients, tools and patterns that connect the service to other systems. |
| Reference | Schemas, settings, compatibility and limits, for looking things up. |
| Operations | What to ask your platform team for (setup or access) and CI/CD and automation, plus security, performance or monitoring where the service needs them. |
| Troubleshooting | Symptoms, causes and fixes, and error messages you can search for. |
Each overview ends with a "What's in this section" (or "How to read these docs") list that links every page.
Conventions you'll meet
- "Applies to VAST Cluster 5.5." Every page states the cluster release it's written for, under its title. Pages that use the VAST DataBase Python SDK also name the SDK version their examples use. See what's new and versions.
- Expected output under code blocks. In quickstarts and guides, the code blocks run in order, and below each one is the output you should see, captured from a 5.5 cluster. Values such as versions, names and times depend on your cluster; the page says so where it matters. Where a block isn't meant to be run on its own, or its output hasn't been captured yet, its first line says so.
- "Admin task: send to your platform team." A step marked like this is done by your platform team, not by you. It's followed by links to the KB procedure, and often by a request you can copy and send.
- Draft pages. A page marked as a draft is under review and may change.
Where to start
- Choose a service for what you want to build. If it spans several services, see end-to-end patterns, or solutions for VAST's AI Factory, cyber lakehouse, analytics and real-time event plays.
- Get access: send your platform team the request on that service's setup or access page.
- Run that service's quickstart, linked from choose a service.
- Choose a client for the language or engine you'll build with.
- Check what's new and versions for your cluster's release and the client versions that go with it.