Publish functions

Applies to VAST Cluster 5.5.

This guide covers getting a function's image from your machine to a running pipeline: how image names, the registry and the function line up, how to register a function, how to release a new version as a new revision, and how a pipeline either follows the latest revision or stays pinned to one. It includes a known vastde issue with updating published functions, and the workaround.

It builds on the quickstart: the thumbnails pipeline runs thumbnailer revision 1, image tag v1. Commands use the environment variables from the quickstart's before you start. The outputs come from a lab cluster running VAST Cluster 5.5.0-SP6 with vastde v5.5.0-sp2, where the registry host is 127.0.0.1:5000, the registry's name is docs-local and VMS is at <VMS address>. Your names, GUIDs, addresses and times will differ.

Name images and tags

Three names have to line up:

Where	Example (lab)	What it is
`vastde functions build <image>`	`127.0.0.1:5000/docs/thumbnailer`	The full image name you push: your registry host, then a path
`--artifact-source`	`docs/thumbnailer`	The same path, without the host and without a tag
`--container-registry`	`docs-local`	The registry's name in DataEngine, whose base URL the cluster puts in front of the path

The cluster pulls <registry base URL>/<artifact source>:<image tag>. Find your registry's name with vastde container-registries list:

vastde container-registries list

Registry Name    Description                       GUID                  Updated at
---------------------------------------------------------------------------------------
docs-local       Local HTTP registry for the d...  1e286592-0291-46e...  2026-10-08 ...

Rules for tags:

A new tag for every build. Each function revision records an image tag, so a reused tag means a revision no longer says which code it runs. Use a version or the Git commit, never latest.
Push before you register. The cluster checks that it can reach the image when you create or update the function, and refuses one it can't find.

Register a function

vastde functions create registers the function and makes revision 1:

vastde functions create --name indexer --description "Indexes thumbnails" --artifact-type image --artifact-source docs/indexer --image-tag v1 --container-registry "$REGISTRY_NAME" --publish

Function created: indexer
Name: indexer
Description: Indexes thumbnails
Tags: []
GUID: 2d2524d6-525f-4e4f-bcdc-f9e2525d4136
Owner: [id: 4294967295, id-type: vid]
Created At: 2026-10-08T12:43:10Z
Updated At: 2026-10-08T12:43:10Z
VRN: vast:dataengine:functions:indexer
Last Revision: 1

--publish publishes the revision. Publish every revision a pipeline will run. The flags can also come from a file (-f); see function and trigger files.

The create fails if anything doesn't line up. A tag or a path that isn't in the registry:

vastde functions create --name ghost --artifact-type image --artifact-source docs/thumbnailer --image-tag v99 --container-registry "$REGISTRY_NAME"

Failed to create function:
400 Bad Request: Image is unreachable: http://<registry host>:5000/docs/thumbnailer:v99, registry: <registry host>:5000, status_code: 404, error: Unknown error, checked_at: 2026-10-08T12:46:36.100985+00:00

The message shows the address the cluster checked, which can differ from the host you push to. A registry name that DataEngine doesn't know:

vastde functions create --name ghost --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry no-such-registry

failed to resolve container registry 'no-such-registry': container registry with name 'no-such-registry' not found

A name that breaks the naming rule, or one that's taken:

vastde functions create --name Ghost_Fn --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry "$REGISTRY_NAME"

Failed to create function:
422 Unprocessable Entity: {"name": "Name must be 2-40 characters: lowercase letters, numbers, and hyphens only."}

vastde functions create --name thumbnailer --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry "$REGISTRY_NAME"

Failed to create function:
409 Conflict: Function with (name=thumbnailer,tenant_guid=da47bae0-aefd-4e85-a5a9-2020d6a2cb91) already exists!

Revisions

vastde functions get <name> -r lists a function's revisions, each with its image tag and whether it's published:

vastde functions get thumbnailer -r

Name: thumbnailer
Description: Makes thumbnails of new images
Tags: []
GUID: 45f7eccb-c794-4fd0-a7df-7abe09386a26
Owner: [id: 4294967295, id-type: vid]
Created At: 2026-10-08T12:24:33Z
Updated At: 2026-10-08T12:24:33Z
VRN: vast:dataengine:functions:thumbnailer
Last Revision: 1

Revisions (1)
  Revision Number: 1
  Description: N/A
  Tags: []
  GUID: 41ba501e-aa46-4454-b51d-c04d507715ff
  Owner: [id: 4294967295, id-type: vid]
  Created At: 2026-10-08T12:24:36Z
  Updated At: 2026-10-08T12:24:36Z
  VRN: vast:dataengine:functions:thumbnailer/1
  Architecture: N/A
  Image Tag: v1
  Container Registry: vast:dataengine:container-registries:docs-local
  Published: true

Each revision has its own VRN, vast:dataengine:functions:<name>/<number>. A new image means a new revision; the old ones stay, so you can go back to them.

Release a new version

Change the code (here, thumbnailer now returns a thumbnail key; the handler is in write a function), and build and push it with a new tag:

vastde functions build "$REGISTRY/docs/thumbnailer" --target ./thumbnailer --image-tag v2 --push

Detected language: python
Validating Python version 3.12.*...
Python version 3.12.* resolved to 3.12.13
Building 127.0.0.1:5000/docs/thumbnailer:v2
App Path: ./thumbnailer
Handlers File: main.py
2026/10/08 12:28:22 [Started] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v2
2026/10/08 12:29:02 [Completed] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v2
Build completed: 127.0.0.1:5000/docs/thumbnailer:v2

Then point the function at the new tag. The command for that is vastde functions update, but see the known issue below first:

vastde functions update thumbnailer --image-tag v2 --revision-description "Return the thumbnail key" --publish

Failed to update function:
422 Unprocessable Entity: {"detail":"{\"last_published_revision_number\": \"Extra inputs are not permitted\"}"}

Known issue: `functions update` fails on published functions

In vastde v5.5.0-sp2, vastde functions update fails with this 422 error for any function that has a published revision. Nothing changes: the function still has only revision 1. The cause is visible with --dry-run: the CLI sends back a read-only field, last_published_revision_number, that the server rejects:

vastde functions update thumbnailer --image-tag v2 --dry-run

[DRY RUN] Would send PUT to /api/latest/serverless/functions/45f7eccb-c794-4fd0-a7df-7abe09386a26
Request Body:
{
  "artifact_source": "docs/thumbnailer",
  "artifact_type": "image",
  "container_registry_vrn": "vast:dataengine:container-registries:docs-local",
  "description": "Makes thumbnails of new images",
  "image_tag": "v2",
  "last_published_revision_number": 1,
  "name": "thumbnailer"
}

functions update works for functions with no published revision.

Workaround: send the same update to the DataEngine API without that field. Get a token for your tenant user (-k skips certificate checks, as on the lab's self-signed VMS; leave it out if your machine trusts VMS's certificate):

export VMS_URL="https://<VMS address>"
export DE_TENANT="<tenant>"
export DE_USER="<user>"
read -rs DE_PASS
export TOKEN=$(curl -sk -X POST "$VMS_URL/api/token/$DE_TENANT" -H "Content-Type: application/json" -d "{\"username\": \"$DE_USER\", \"password\": \"$DE_PASS\"}" | jq -r .access)
FUNCTION_GUID=$(vastde functions get thumbnailer -o json | jq -r .guid)

Write the function's full definition, with the new tag and is_published, to update-v2.json:

{
  "name": "thumbnailer",
  "description": "Makes thumbnails of new images",
  "artifact_type": "image",
  "artifact_source": "docs/thumbnailer",
  "image_tag": "v2",
  "container_registry_vrn": "vast:dataengine:container-registries:docs-local",
  "revision_description": "Return the thumbnail key",
  "is_published": true
}

Send it:

curl -sk -X PUT "$VMS_URL/api/latest/serverless/functions/$FUNCTION_GUID" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @update-v2.json | jq .

{
  "default_revision_number": null,
  "name": "thumbnailer",
  "description": "Makes thumbnails of new images",
  "tags": null,
  "id": 7513937380001185792,
  "guid": "45f7eccb-c794-4fd0-a7df-7abe09386a26",
  "tenant_guid": "da47bae0-aefd-4e85-a5a9-2020d6a2cb91",
  "owner": {
    "id": "4294967295",
    "id_type": "vid",
    "name": "root"
  },
  "created_at": "2026-10-08T12:24:33.598000Z",
  "updated_at": "2026-10-08T12:34:43.532383Z",
  "vrn": "vast:dataengine:functions:thumbnailer",
  "last_revision_number": 2,
  "last_published_revision_number": 1,
  "container_registry_vrn": "vast:dataengine:container-registries:docs-local"
}

last_revision_number is now 2. Check the revisions:

vastde functions get thumbnailer -r

Name: thumbnailer
Description: Makes thumbnails of new images
Tags: []
GUID: 45f7eccb-c794-4fd0-a7df-7abe09386a26
Owner: [id: 4294967295, id-type: vid]
Created At: 2026-10-08T12:24:33Z
Updated At: 2026-10-08T12:34:43Z
VRN: vast:dataengine:functions:thumbnailer
Last Revision: 2

Revisions (2)
  Revision Number: 1
  Description: N/A
  Tags: []
  GUID: 41ba501e-aa46-4454-b51d-c04d507715ff
  Owner: [id: 4294967295, id-type: vid]
  Created At: 2026-10-08T12:24:36Z
  Updated At: 2026-10-08T12:24:36Z
  VRN: vast:dataengine:functions:thumbnailer/1
  Architecture: N/A
  Image Tag: v1
  Container Registry: vast:dataengine:container-registries:docs-local
  Published: true
  Revision Number: 2
  Description: Return the thumbnail key
  Tags: []
  GUID: d0b4ff73-0455-4729-a463-d8a1eeea8c09
  Owner: [id: 4294967295, id-type: vid]
  Created At: 2026-10-08T12:34:43Z
  Updated At: 2026-10-08T12:34:43Z
  VRN: vast:dataengine:functions:thumbnailer/2
  Architecture: N/A
  Image Tag: v2
  Container Registry: vast:dataengine:container-registries:docs-local
  Published: true

Revision 2 is published and points at v2. In CI, keep the JSON body in Git next to the code and change only image_tag and revision_description per release. Each update makes a new revision, so send it once per release.

Which revision a pipeline runs

A new revision doesn't change a running pipeline by itself. The pipeline file decides, per function deployment:

Unpinned (no revision, as in the quickstart): the deployment runs the function's default revision, which is its latest revision unless you set a default. The revision is chosen when the pipeline is deployed.
Pinned (revision: <number>): the deployment runs that revision until you change the file and deploy again.

See which revision runs.

Unpinned: deploy to pick up the new revision

Right after the update, the pipeline still runs revision 1. An upload is still handled by the old code (its log line has no v2:):

aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp dog.jpg "s3://$BUCKET/photos/dog.jpg"

Completed 132 Bytes/132 Bytes (5.2 KiB/s) with 1 file(s) remaining
upload: ./dog.jpg to s3://docs-src/photos/dog.jpg

vastde logs get thumbnails --since 2m --scope user

2026-10-08 12:35:07.07 [thumbnailer] [INFO]  [user] new object: bucket=docs-src key=photos/dog.jpg

Deploy the pipeline again, with no change to its file:

vastde pipelines deploy thumbnails

Pipeline deployed successfully

Wait for vastde pipelines get thumbnails to show Status: Ready again (it shows InProgress first), then upload another object:

aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp owl.jpg "s3://$BUCKET/photos/owl.jpg"

Completed 132 Bytes/132 Bytes (4.1 KiB/s) with 1 file(s) remaining
upload: ./owl.jpg to s3://docs-src/photos/owl.jpg

vastde logs get thumbnails --since 1m --scope user

2026-10-08 12:36:10.25 [thumbnailer] [INFO]  [user] thumbnailer v2 initialized
2026-10-08 12:36:48.61 [thumbnailer] [INFO]  [user] v2: thumbnail for s3://docs-src/photos/owl.jpg -> thumbnails/owl.jpg

A new instance started with revision 2 and handled the upload.

Because an unpinned deployment takes whatever is latest at deploy time, anyone who deploys the pipeline for any reason also rolls out every function's newest revision. That's convenient in development, but in a pipeline you promote between environments, pin.

Pinned: run exactly the revision you tested

Add revision to the function deployment in pipeline.yaml. This pins thumbnailer back to revision 1:

name: thumbnails
description: Make a thumbnail for every new image in docs-src/photos/
kubernetes_cluster_vrn: vast:dataengine:kubernetes-clusters:test-cluster
namespace: dataengine-ns-da47bae0-aefd-4e85-a5a9-2020d6a2cb91
manifest:
  triggers:
    - name: uploads
      vrn: vast:dataengine:triggers:docs-src-uploads
  function_deployments:
    - name: thumbnailer
      function_vrn: vast:dataengine:functions:thumbnailer
      revision: 1
  links:
    - source: [uploads]
      destination: [thumbnailer]

vastde pipelines update thumbnails --config @pipeline.yaml

Pipeline updated successfully
Name: thumbnails
Description: Make a thumbnail for every new image in docs-src/photos/
Tags: []
GUID: c7f63133-3438-4cae-af6a-91291041f1b6
Owner: [id: 4294967295, id-type: vid]
Created At: 2026-10-08T12:25:11Z
Updated At: 2026-10-08T12:37:12Z
VRN: vast:dataengine:pipelines:thumbnails
Status: Ready

The update saves the change as the pipeline's latest revision but doesn't deploy it. The latest manifest is pinned; the deployed one isn't, yet:

vastde pipelines get thumbnails --manifest -o yaml | grep revision

      revision: 1

vastde pipelines get thumbnails --deployed -o yaml | grep revision

      revision: null

Deploy it:

vastde pipelines deploy thumbnails

Pipeline deployed successfully

Once it's Ready, uploads run revision 1 again (its init message is thumbnailer initialized, and its log line has no v2:):

aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp fox.jpg "s3://$BUCKET/photos/fox.jpg"

Completed 132 Bytes/132 Bytes (3.6 KiB/s) with 1 file(s) remaining
upload: ./fox.jpg to s3://docs-src/photos/fox.jpg

vastde logs get thumbnails --since 1m --scope user

2026-10-08 12:38:00.46 [thumbnailer] [INFO]  [user] thumbnailer initialized
2026-10-08 12:38:36.15 [thumbnailer] [INFO]  [user] new object: bucket=docs-src key=photos/fox.jpg

This is also how you roll back a bad release: pin the previous revision, update and deploy. To follow the latest revision again, remove the revision line, update and deploy. The rest of this section's guides assume thumbnailer is unpinned and on revision 2.

A release, step by step

Change the code, and test it locally (build and test locally).
vastde functions build <registry host>/<path> --image-tag <new tag> --push.
Point the function at the new tag (with the workaround above while the CLI issue lasts), published. Note the new revision number.
If the deployment is pinned, change revision in the pipeline file and commit it.
vastde pipelines update <pipeline> --config @pipeline.yaml, then vastde pipelines deploy <pipeline>, and wait for Ready.

Pipelines as code covers the pipeline side.