--- title: "Publish functions" slug: "dev-dataengine-guides-publish-functions" description: "Learn how to manage function images, revisions, and pipelines in VAST Cluster, including registration, updates, and known issues with published functions." tags: ["DataEngine"] updated: 2026-10-28T02:14:59Z published: 2026-10-28T02:14:59Z canonical: "docs.vastdata.com/dev-dataengine-guides-publish-functions" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Publish functions *Applies to VAST Cluster 5.5.* This guide covers getting a function's image from your machine to a running pipeline: how image names, the registry and the function line up, how to register a function, how to release a new version as a new revision, and how a pipeline either follows the latest revision or stays pinned to one. It includes a known `vastde` issue with updating published functions, and the workaround. It builds on the [quickstart](/develop/docs/dev-dataengine-quickstart): the `thumbnails` pipeline runs `thumbnailer` revision 1, image tag `v1`. Commands use the environment variables from the quickstart's [before you start](/develop/docs/dev-dataengine-quickstart#before-you-start). The outputs come from a lab cluster running VAST Cluster 5.5.0-SP6 with `vastde` v5.5.0-sp2, where the registry host is `127.0.0.1:5000`, the registry's name is `docs-local` and VMS is at `<VMS address>`. Your names, GUIDs, addresses and times will differ. ## Name images and tags Three names have to line up: | Where | Example (lab) | What it is | | --- | --- | --- | | `vastde functions build <image>` | `127.0.0.1:5000/docs/thumbnailer` | The full image name you push: your registry host, then a path | | `--artifact-source` | `docs/thumbnailer` | The same path, without the host and without a tag | | `--container-registry` | `docs-local` | The registry's name in DataEngine, whose base URL the cluster puts in front of the path | The cluster pulls `<registry base URL>/<artifact source>:<image tag>`. Find your registry's name with `vastde container-registries list`: ``` vastde container-registries list ``` ``` Registry Name Description GUID Updated at --------------------------------------------------------------------------------------- docs-local Local HTTP registry for the d... 1e286592-0291-46e... 2026-10-08 ... ``` Rules for tags: - **A new tag for every build.** Each function revision records an image tag, so a reused tag means a revision no longer says which code it runs. Use a version or the Git commit, never `latest`. - **Push before you register.** The cluster checks that it can reach the image when you create or update the function, and refuses one it can't find. ## Register a function `vastde functions create` registers the function and makes revision 1: ``` vastde functions create --name indexer --description "Indexes thumbnails" --artifact-type image --artifact-source docs/indexer --image-tag v1 --container-registry "$REGISTRY_NAME" --publish ``` ``` Function created: indexer Name: indexer Description: Indexes thumbnails Tags: [] GUID: 2d2524d6-525f-4e4f-bcdc-f9e2525d4136 Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:43:10Z Updated At: 2026-10-08T12:43:10Z VRN: vast:dataengine:functions:indexer Last Revision: 1 ``` `--publish` publishes the revision. Publish every revision a pipeline will run. The flags can also come from a file (`-f`); see [function and trigger files](/develop/docs/dev-dataengine-reference-function-and-trigger-files#function-file). The create fails if anything doesn't line up. A tag or a path that isn't in the registry: ``` vastde functions create --name ghost --artifact-type image --artifact-source docs/thumbnailer --image-tag v99 --container-registry "$REGISTRY_NAME" ``` ``` Failed to create function: 400 Bad Request: Image is unreachable: http://:5000/docs/thumbnailer:v99, registry: :5000, status_code: 404, error: Unknown error, checked_at: 2026-10-08T12:46:36.100985+00:00 ``` The message shows the address the cluster checked, which can differ from the host you push to. A registry name that DataEngine doesn't know: ``` vastde functions create --name ghost --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry no-such-registry ``` ``` failed to resolve container registry 'no-such-registry': container registry with name 'no-such-registry' not found ``` A name that breaks the naming rule, or one that's taken: ``` vastde functions create --name Ghost_Fn --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry "$REGISTRY_NAME" ``` ``` Failed to create function: 422 Unprocessable Entity: {"name": "Name must be 2-40 characters: lowercase letters, numbers, and hyphens only."} ``` ``` vastde functions create --name thumbnailer --artifact-type image --artifact-source docs/thumbnailer --image-tag v1 --container-registry "$REGISTRY_NAME" ``` ``` Failed to create function: 409 Conflict: Function with (name=thumbnailer,tenant_guid=da47bae0-aefd-4e85-a5a9-2020d6a2cb91) already exists! ``` ## Revisions `vastde functions get <name> -r` lists a function's revisions, each with its image tag and whether it's published: ``` vastde functions get thumbnailer -r ``` ``` Name: thumbnailer Description: Makes thumbnails of new images Tags: [] GUID: 45f7eccb-c794-4fd0-a7df-7abe09386a26 Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:24:33Z Updated At: 2026-10-08T12:24:33Z VRN: vast:dataengine:functions:thumbnailer Last Revision: 1 Revisions (1) Revision Number: 1 Description: N/A Tags: [] GUID: 41ba501e-aa46-4454-b51d-c04d507715ff Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:24:36Z Updated At: 2026-10-08T12:24:36Z VRN: vast:dataengine:functions:thumbnailer/1 Architecture: N/A Image Tag: v1 Container Registry: vast:dataengine:container-registries:docs-local Published: true ``` Each revision has its own VRN, `vast:dataengine:functions:<name>/<number>`. A new image means a new revision; the old ones stay, so you can go back to them. ## Release a new version Change the code (here, `thumbnailer` now returns a thumbnail key; the handler is in [write a function](/develop/docs/dev-dataengine-guides-write-a-function#return-values-and-chaining)), and build and push it with a new tag: ``` vastde functions build "$REGISTRY/docs/thumbnailer" --target ./thumbnailer --image-tag v2 --push ``` ``` Detected language: python Validating Python version 3.12.*... Python version 3.12.* resolved to 3.12.13 Building 127.0.0.1:5000/docs/thumbnailer:v2 App Path: ./thumbnailer Handlers File: main.py 2026/10/08 12:28:22 [Started] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v2 2026/10/08 12:29:02 [Completed] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v2 Build completed: 127.0.0.1:5000/docs/thumbnailer:v2 ``` Then point the function at the new tag. The command for that is `vastde functions update`, but see the known issue below first: ``` vastde functions update thumbnailer --image-tag v2 --revision-description "Return the thumbnail key" --publish ``` ``` Failed to update function: 422 Unprocessable Entity: {"detail":"{\"last_published_revision_number\": \"Extra inputs are not permitted\"}"} ``` ### Known issue: `functions update` fails on published functions In `vastde` v5.5.0-sp2, `vastde functions update` fails with this 422 error for any function that has a published revision. Nothing changes: the function still has only revision 1. The cause is visible with `--dry-run`: the CLI sends back a read-only field, `last_published_revision_number`, that the server rejects: ``` vastde functions update thumbnailer --image-tag v2 --dry-run ``` ``` [DRY RUN] Would send PUT to /api/latest/serverless/functions/45f7eccb-c794-4fd0-a7df-7abe09386a26 Request Body: { "artifact_source": "docs/thumbnailer", "artifact_type": "image", "container_registry_vrn": "vast:dataengine:container-registries:docs-local", "description": "Makes thumbnails of new images", "image_tag": "v2", "last_published_revision_number": 1, "name": "thumbnailer" } ``` `functions update` works for functions with no published revision. **Workaround: send the same update to the DataEngine API without that field.** Get a token for your tenant user (`-k` skips certificate checks, as on the lab's self-signed VMS; leave it out if your machine trusts VMS's certificate): ``` export VMS_URL="https://" export DE_TENANT="" export DE_USER="" read -rs DE_PASS export TOKEN=$(curl -sk -X POST "$VMS_URL/api/token/$DE_TENANT" -H "Content-Type: application/json" -d "{\"username\": \"$DE_USER\", \"password\": \"$DE_PASS\"}" | jq -r .access) FUNCTION_GUID=$(vastde functions get thumbnailer -o json | jq -r .guid) ``` Write the function's full definition, with the new tag and `is_published`, to `update-v2.json`: ``` { "name": "thumbnailer", "description": "Makes thumbnails of new images", "artifact_type": "image", "artifact_source": "docs/thumbnailer", "image_tag": "v2", "container_registry_vrn": "vast:dataengine:container-registries:docs-local", "revision_description": "Return the thumbnail key", "is_published": true } ``` Send it: ``` curl -sk -X PUT "$VMS_URL/api/latest/serverless/functions/$FUNCTION_GUID" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @update-v2.json | jq . ``` ``` { "default_revision_number": null, "name": "thumbnailer", "description": "Makes thumbnails of new images", "tags": null, "id": 7513937380001185792, "guid": "45f7eccb-c794-4fd0-a7df-7abe09386a26", "tenant_guid": "da47bae0-aefd-4e85-a5a9-2020d6a2cb91", "owner": { "id": "4294967295", "id_type": "vid", "name": "root" }, "created_at": "2026-10-08T12:24:33.598000Z", "updated_at": "2026-10-08T12:34:43.532383Z", "vrn": "vast:dataengine:functions:thumbnailer", "last_revision_number": 2, "last_published_revision_number": 1, "container_registry_vrn": "vast:dataengine:container-registries:docs-local" } ``` `last_revision_number` is now 2. Check the revisions: ``` vastde functions get thumbnailer -r ``` ``` Name: thumbnailer Description: Makes thumbnails of new images Tags: [] GUID: 45f7eccb-c794-4fd0-a7df-7abe09386a26 Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:24:33Z Updated At: 2026-10-08T12:34:43Z VRN: vast:dataengine:functions:thumbnailer Last Revision: 2 Revisions (2) Revision Number: 1 Description: N/A Tags: [] GUID: 41ba501e-aa46-4454-b51d-c04d507715ff Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:24:36Z Updated At: 2026-10-08T12:24:36Z VRN: vast:dataengine:functions:thumbnailer/1 Architecture: N/A Image Tag: v1 Container Registry: vast:dataengine:container-registries:docs-local Published: true Revision Number: 2 Description: Return the thumbnail key Tags: [] GUID: d0b4ff73-0455-4729-a463-d8a1eeea8c09 Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:34:43Z Updated At: 2026-10-08T12:34:43Z VRN: vast:dataengine:functions:thumbnailer/2 Architecture: N/A Image Tag: v2 Container Registry: vast:dataengine:container-registries:docs-local Published: true ``` Revision 2 is published and points at `v2`. In CI, keep the JSON body in Git next to the code and change only `image_tag` and `revision_description` per release. Each update makes a new revision, so send it once per release. ## Which revision a pipeline runs A new revision doesn't change a running pipeline by itself. The pipeline file decides, per function deployment: - **Unpinned** (no `revision`, as in the quickstart): the deployment runs the function's default revision, which is its latest revision unless you set a default. The revision is chosen **when the pipeline is deployed**. - **Pinned** (`revision: <number>`): the deployment runs that revision until you change the file and deploy again. See [which revision runs](/develop/docs/dev-dataengine-concepts-index#which-revision-runs). ### Unpinned: deploy to pick up the new revision Right after the update, the pipeline still runs revision 1. An upload is still handled by the old code (its log line has no `v2:`): ``` aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp dog.jpg "s3://$BUCKET/photos/dog.jpg" ``` ``` Completed 132 Bytes/132 Bytes (5.2 KiB/s) with 1 file(s) remaining upload: ./dog.jpg to s3://docs-src/photos/dog.jpg ``` ``` vastde logs get thumbnails --since 2m --scope user ``` ``` 2026-10-08 12:35:07.07 [thumbnailer] [INFO] [user] new object: bucket=docs-src key=photos/dog.jpg ``` Deploy the pipeline again, with no change to its file: ``` vastde pipelines deploy thumbnails ``` ``` Pipeline deployed successfully ``` Wait for `vastde pipelines get thumbnails` to show `Status: Ready` again (it shows `InProgress` first), then upload another object: ``` aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp owl.jpg "s3://$BUCKET/photos/owl.jpg" ``` ``` Completed 132 Bytes/132 Bytes (4.1 KiB/s) with 1 file(s) remaining upload: ./owl.jpg to s3://docs-src/photos/owl.jpg ``` ``` vastde logs get thumbnails --since 1m --scope user ``` ``` 2026-10-08 12:36:10.25 [thumbnailer] [INFO] [user] thumbnailer v2 initialized 2026-10-08 12:36:48.61 [thumbnailer] [INFO] [user] v2: thumbnail for s3://docs-src/photos/owl.jpg -> thumbnails/owl.jpg ``` A new instance started with revision 2 and handled the upload. Because an unpinned deployment takes whatever is latest at deploy time, anyone who deploys the pipeline for any reason also rolls out every function's newest revision. That's convenient in development, but in a pipeline you promote between environments, pin. ### Pinned: run exactly the revision you tested Add `revision` to the function deployment in `pipeline.yaml`. This pins `thumbnailer` back to revision 1: ``` name: thumbnails description: Make a thumbnail for every new image in docs-src/photos/ kubernetes_cluster_vrn: vast:dataengine:kubernetes-clusters:test-cluster namespace: dataengine-ns-da47bae0-aefd-4e85-a5a9-2020d6a2cb91 manifest: triggers: - name: uploads vrn: vast:dataengine:triggers:docs-src-uploads function_deployments: - name: thumbnailer function_vrn: vast:dataengine:functions:thumbnailer revision: 1 links: - source: [uploads] destination: [thumbnailer] ``` ``` vastde pipelines update thumbnails --config @pipeline.yaml ``` ``` Pipeline updated successfully Name: thumbnails Description: Make a thumbnail for every new image in docs-src/photos/ Tags: [] GUID: c7f63133-3438-4cae-af6a-91291041f1b6 Owner: [id: 4294967295, id-type: vid] Created At: 2026-10-08T12:25:11Z Updated At: 2026-10-08T12:37:12Z VRN: vast:dataengine:pipelines:thumbnails Status: Ready ``` The update saves the change as the pipeline's latest revision but doesn't deploy it. The latest manifest is pinned; the deployed one isn't, yet: ``` vastde pipelines get thumbnails --manifest -o yaml | grep revision ``` ``` revision: 1 ``` ``` vastde pipelines get thumbnails --deployed -o yaml | grep revision ``` ``` revision: null ``` Deploy it: ``` vastde pipelines deploy thumbnails ``` ``` Pipeline deployed successfully ``` Once it's `Ready`, uploads run revision 1 again (its `init` message is `thumbnailer initialized`, and its log line has no `v2:`): ``` aws --region us-east-1 --endpoint-url "$S3_ENDPOINT" s3 cp fox.jpg "s3://$BUCKET/photos/fox.jpg" ``` ``` Completed 132 Bytes/132 Bytes (3.6 KiB/s) with 1 file(s) remaining upload: ./fox.jpg to s3://docs-src/photos/fox.jpg ``` ``` vastde logs get thumbnails --since 1m --scope user ``` ``` 2026-10-08 12:38:00.46 [thumbnailer] [INFO] [user] thumbnailer initialized 2026-10-08 12:38:36.15 [thumbnailer] [INFO] [user] new object: bucket=docs-src key=photos/fox.jpg ``` This is also how you roll back a bad release: pin the previous revision, update and deploy. To follow the latest revision again, remove the `revision` line, update and deploy. The rest of this section's guides assume `thumbnailer` is unpinned and on revision 2. ## A release, step by step 1. Change the code, and test it locally ([build and test locally](/develop/docs/dev-dataengine-guides-build-and-test-locally)). 2. `vastde functions build <registry host>/<path> --image-tag <new tag> --push`. 3. Point the function at the new tag (with the workaround above while the CLI issue lasts), published. Note the new revision number. 4. If the deployment is pinned, change `revision` in the pipeline file and commit it. 5. `vastde pipelines update <pipeline> --config @pipeline.yaml`, then `vastde pipelines deploy <pipeline>`, and wait for `Ready`. [Pipelines as code](/develop/docs/dev-dataengine-guides-pipelines-as-code) covers the pipeline side. ## See also - [Concepts: functions and revisions](/develop/docs/dev-dataengine-concepts-index#functions-and-revisions) - [Function and trigger files](/develop/docs/dev-dataengine-reference-function-and-trigger-files): every field of a function, and when a revision is created. - [Pipeline YAML: pinning a revision](/develop/docs/dev-dataengine-reference-pipeline-yaml#pinning-a-revision) - [Setup prerequisites](/develop/docs/dev-dataengine-operations-setup-prerequisites): the registry your platform team links. - KB: [Managing DataEngine functions](/docs-5-5/docs/managing-dataengine-functions)