Coming from the Web UI

Applies to VAST Cluster 5.5.

If your team started with VAST DataEngine in the VAST Web UI, with its forms and the Visual Builder, this page maps what you know to the code-first workflow. For every UI form it gives the vastde flag and the YAML key that set the same thing. It then covers the names that differ, how to move a pipeline you built in the UI into Git, and why one pipeline should have one place where it's edited.

The UI and the CLI write the same objects through the same API. Nothing you built in the UI has to be rebuilt; it has to be exported once and then edited only from its files.

Where the UI fits in a code-first workflow

Triggers

Create Trigger form. Flags are for vastde triggers create element or triggers create schedule; keys are for the trigger file.

In 5.5, removing a tag in the UI's Update Trigger dialog doesn't take effect. Use vastde triggers update <name> --tags ….

Functions

Create Function and Edit Function forms. Flags are for vastde functions create and functions update; keys are for the function file.

Saving the Edit Function form creates a new revision. So does functions update when the image, registry, architecture, publish state, tags or alias change. See revisions.

Pipelines

Create Pipeline form. Keys are for the pipeline file.

The UI's import files are not the CLI's. The UI's secret import is a flat map of keys under one top-level key, and Import ConfigMap is a flat KEY: "value" map. The CLI's --secret-file holds name, kubernetes_cluster_vrn, namespace and entries: [{key, value}], and environment variables in the pipeline file are a list of {name, value}.

Visual Builder

Function deployments

The function deployment configuration panel in the Visual Builder. Keys are under the deployment's entry in manifest.function_deployments, except where noted.

Names that differ

Moving a UI-built pipeline into Git

Do this once per pipeline. <!-- TODO: verify on devvm: the UI Download and steps 2-3 haven't been exercised; step 4's get --manifest has -->

1. Export the pipeline. In the Visual Builder's YAML view, click Download to save pipeline-manifest.yaml. That's a complete pipeline file. Or build the file from the CLI as described in exporting an existing pipeline.
2. Write the trigger and function files. There's no export for these. Use vastde triggers get <name> -o yaml and vastde functions get <name> -r -o yaml to read the current values, and write trigger and function files from them. Don't copy read-only fields such as guid, vrn or status.
3. Pin revisions. Set revision on every function deployment to the revision that's running now, so that the file means exactly what's deployed. vastde pipelines get <name> --manifest --deployed -o yaml shows the deployed manifest.
4. Check it applies. Run vastde pipelines update <name> --config <file> and then vastde pipelines get <name> --manifest -o yaml, and compare it with your file's manifest section. Expect differences you didn't write: the cluster fills in a resources block (scaling, CPU and memory) for each function deployment, shows revision: null on unpinned ones, and leaves out an empty config. If the latest revision was deployed, the update creates a new revision; it doesn't change the running pipeline until you deploy.
5. Commit the files. Leave secret files out of Git.
6. Switch the team over. From now on, change the pipeline only by editing its files and running update and deploy, by hand or from CI. Use the UI to watch it.

One source of truth

The UI and the CLI save pipelines through the same API, and that API has no version check: no ETag, no "this pipeline changed since you loaded it". The last writer wins, section by section of the manifest.

• If someone edits a pipeline in the Visual Builder and saves, and someone else then runs pipelines update with the file from Git, the file's sections replace the UI's edits without a warning.
• The other way round, a UI save replaces what the file put there. The next update from Git puts it back.
• A section the file leaves out is kept, so a stale or partial file can leave a mix of both.
• Saving in the UI doesn't deploy, but it can create a new revision, and the next deploy, from either side, deploys the latest revision.

So pick one place to edit each pipeline. For pipelines you manage from Git, treat the UI as read-only: observe, inspect, read logs and traces, but don't save.

Registries and compute clusters

These forms are usually the platform team's work, not yours: linking a container registry, linking a Kubernetes cluster, creating a VAST compute cluster. If you need one, see setup prerequisites for what to ask for. The CLI equivalents are listed here so that you can read a setup script. In 5.5 these commands take flags only, not a file.

Container registry

Link a container registry form, vastde container-registries link:

A pipeline can deploy a function only if the function's registry is linked to the pipeline's compute cluster and namespace. See what a deploy requires.

External Kubernetes cluster

Link Kubernetes Cluster form, vastde compute-clusters link:

The cluster's VRN, which pipeline files use, is vast:dataengine:kubernetes-clusters:<name>.

VAST compute cluster

Creating a VAST compute cluster (Kubernetes on the cluster's own CNodes) is a cluster-admin task in the VAST Web UI or VAST CLI, with no vastde equivalent. When the platform team attaches your tenant to it, the cluster is linked to your tenant and gets your tenant's DataEngine namespace; you don't run vastde compute-clusters link for it. vastde compute-clusters list shows it. See Managing VAST Compute Clusters on the KB.

See also

• Pipeline YAML
• Function and trigger files
• Pipelines as code
• CI/CD
• Setup prerequisites
• Building and Deploying a Pipeline on VAST DataEngine, Creating a Trigger and Managing DataEngine Functions on the KB