Welcome to the new DOCS.vastdata.com. We've added new menus and a version selector, plus new persona and resource pages to help customers get started. Looking for a previous article? Use the docs search.

Solutions

Prev Next

Applies to VAST Cluster 5.5.

Solutions are VAST's plays for a business goal, such as an AI factory, a cyber lakehouse or a neocloud. Each one below links to VAST's own page for it, then lists what to read in these docs to build it, in order. Most steps are end-to-end patterns: step-by-step recipes for combining two or three services.

Enterprise AI Factory

Run AI pipelines next to your data: ingest documents and media, embed them, retrieve context and serve it to models, on one platform. VAST: Build an Enterprise AI Factory at Scale.

Documents and media land in a bucket on VAST DataStore. A VAST DataEngine pipeline chunks and embeds them. The embeddings go into a vector-indexed table in VAST Vector Search, which retrieves the nearest chunks the user may read. Your application passes them to your model to generate the answer.

In these docs:

  1. Retrieval for RAG, with permissions: chunks, embeddings and permission-aware retrieval.
  2. Process new objects into a table: run a function on every new object.
  3. VAST Vector Search and VAST DataEngine: the two services underneath.
  4. Reference applications: a RAG research assistant and a video search application built this way.

Cyber lakehouse

Keep security logs and their long-term history on VAST, and query them with SQL, alongside the tools you use for real-time alerting. VAST: VAST for Cybersecurity (solution brief), the next-generation cyber lakehouse (white paper), Splunk.

Your log sources send events to a VAST Event Broker topic. Blob expansion turns each JSON event into a typed row of a VAST DataBase table. Log files, batch or historical, land as Parquet files on VAST DataStore and are imported into tables. You query the tables with Trino, Spark or SQL, and the VAST Catalog audit log records who accessed what on the cluster. Shipping logs from a log pipeline or a SIEM isn't covered in these docs.

In these docs:
Logs can come in two ways, and you can use either or both:

  1. Stream JSON events into a table: log events through a Kafka topic, with blob expansion turning each JSON event into a typed row as it arrives.
  2. Land files and query them with SQL: log files, batch or historical, landed as Parquet and imported into tables.
  3. Trino and Spark: ad hoc and large-scale queries over the tables.
  4. Find and audit data: who accessed what on the cluster itself.

Not covered in these docs: shipping logs into VAST from a log pipeline or a SIEM.

Data analytics and lakehouse

Run SQL and Spark on tables and files that live on one platform, with the query engine of your choice. VAST: Data Analytics, with Spark, Starburst, Dremio and Vertica.

Parquet files land in a bucket on VAST DataStore and are imported into VAST DataBase tables. You design the tables for your queries, query them with Trino, Spark or the VAST Query Engine, and protect them with snapshots and replication.

In these docs:

  1. Land files and query them with SQL: import Parquet into tables.
  2. Data modeling and querying: table design and the query engines.
  3. Trino and Spark: connect the engines.
  4. Snapshots and replication: point-in-time copies and disaster recovery for your tables.

Real-time event analytics

Query streaming events with SQL as they arrive, without a job that copies them into tables. VAST: Real-Time Event Analytics.

Your producers write events with Kafka clients to a VAST Event Broker topic, which is also a table. Blob expansion turns JSON events into typed rows as they're produced. You query the events with SQL as they arrive, and can fold an event log into a current-state table.

In these docs:

  1. Stream JSON events into a table: events as typed rows, in the same transaction as the produce.
  2. Query topics as tables: read a topic's raw events with SQL.
  3. Event sourcing and CQRS: an event log and a current-state table built from it.
  4. VAST Event Broker: the service underneath.

Neoclouds

Run many customers' AI workloads on one VAST cluster, each in its own tenant with its own network, users, keys, capacity and performance limits. VAST: Building AI Factories: a blueprint for next-gen GPU clouds (blog).

A cluster admin creates a tenant for each customer, with its own virtual IP pools. The customer's tenant admin creates the users, keys, databases, buckets and Kafka views in it. Quotas and QoS policies set the tenant's capacity and performance limits. Automation tools create each tenant's objects from code, and the tenant runs its AI workloads, such as an AI factory pipeline, on the shared cluster.

This path is for developers who build on a neocloud tenant, or who build the provider's own tenant-facing services. Creating and administering tenants is a cluster admin's job, in the KB: creating tenants and managing tenant administrators.

In these docs:

  1. Tenants and one Element Store: what a tenant owns and what it shares with the rest of the cluster.
  2. Developer tenant: a tenant admin sets up users, keys, databases, buckets and Kafka views inside one tenant. The same steps work for any tenant.
  3. Quotas and quality of service: the capacity and performance limits a tenant runs under.
  4. Provision and deploy from CI and automation tools: create each tenant's objects from code.
  5. Enterprise AI Factory: the workloads tenants run on top, and the VAST Catalog access page for what each tenant can see in the Catalog.

See also