--- title: "Build and test locally" slug: "dev-dataengine-guides-build-and-test-locally" description: "Learn to build, run, and test your function projects locally with vastde functions, ensuring error-free deployment before reaching the cluster." tags: ["DataEngine"] updated: 2026-10-28T02:14:59Z published: 2026-10-28T02:14:59Z canonical: "docs.vastdata.com/dev-dataengine-guides-build-and-test-locally" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Build and test locally *Applies to VAST Cluster 5.5.* This guide shows how to turn a function project into an image with `vastde functions build`, run that image on your machine with `vastde functions localrun`, and send it events with `vastde functions invoke`, so you find most mistakes before anything reaches the cluster. It ends with what a local run doesn't emulate. You need Docker running and the `vastde` CLI configured; see [your workstation](/develop/docs/dev-dataengine-operations-setup-prerequisites#your-workstation). Commands use the environment variables from the [quickstart](/develop/docs/dev-dataengine-quickstart#before-you-start). The outputs come from a lab machine with VAST Cluster 5.5.0-SP6 and `vastde` v5.5.0-sp2, where the registry host is `127.0.0.1:5000`. Your paths, IDs and times will differ. ## Build an image `vastde functions build` builds an image from a function project with the builder image, and `--push` pushes it to your registry: ``` vastde functions build "$REGISTRY/docs/thumbnailer" --target ./thumbnailer --image-tag v1 --push ``` ``` Detected language: python Validating Python version 3.12.*... Python version 3.12.* resolved to 3.12.13 Building 127.0.0.1:5000/docs/thumbnailer:v1 App Path: ./thumbnailer Handlers File: main.py 2026/10/08 12:20:16 [Started] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v1 2026/10/08 12:21:00 [Completed] Python Builder: 127.0.0.1:5000/docs/thumbnailer:v1 Build completed: 127.0.0.1:5000/docs/thumbnailer:v1 ``` - **The image name** is `<registry host>/<path>`; `--image-tag` sets the tag. Use a new tag for every build, so that each function revision points at exactly one image. See [publish functions](/develop/docs/dev-dataengine-guides-publish-functions#name-images-and-tags). - **`--target`** is the project directory. The handler file is `main.py` unless you pass `-H` (`--handlers`). - **Python version.** The build defaults to Python 3.12 and resolves it to a patch release, as above. - **Network access.** The build pulls a base image from Docker Hub and installs your `requirements.txt` from PyPI, so the machine needs access to both, or to mirrors. - **`--push`** uses your own Docker login for the registry (`docker login`), not DataEngine's registry settings. The build checks the handler file before it builds anything. A file without `init` fails, and so does a `--target` with no handler file in it: ``` vastde functions build "$REGISTRY/docs/noinit" --target ./does-not-exist --image-tag v1 ``` ``` Detected language: python Validation failed: handler file not found: does-not-exist/main.py handler file not found: does-not-exist/main.py ``` The `init` case is in [write a function](/develop/docs/dev-dataengine-guides-write-a-function#the-handler-file). ### The builder image The builder is an image too. By default the CLI uses the builder whose tag matches its own version. `vastde builders list` shows the published tags: ``` vastde builders list ``` ``` Fetching available builder images from Docker Hub... REPOSITORY TAG SIZE CREATED -------------------------------------------------------------------------------------------- vastdataorg/vast-builder latest 700.9 MB 1 month ago vastdataorg/vast-builder v5.5.0-sp2 700.9 MB 1 month ago vastdataorg/vast-builder v5.4.6-hf6 700.0 MB 1 month ago vastdataorg/vast-builder v5.4.3-sp4-hf21 700.0 MB 1 month ago vastdataorg/vast-builder v5.5.0 700.0 MB 1 month ago … To set a builder image, use: vastde builders set ``` Use the tag your platform team names for your cluster, and set it with `vastde builders set <tag>`; `vastde config view` shows the one in use. Don't use `latest` or the `dev` and `rc` tags: pin a release tag so that builds are repeatable. `vastde builders inspect <tag>` shows a tag's digest and platform (`linux/amd64`). ## Run it locally `vastde functions localrun` runs the image in a local Docker container, with the DataEngine runtime listening on the port you choose. `--detach` runs it in the background: ``` vastde functions localrun "$REGISTRY/docs/thumbnailer" --image-tag v1 --port 18080 --detach ``` ``` Running function '127.0.0.1:5000/docs/thumbnailer:v1' as Docker container on port 18080 Function is running in detached mode on port 18080! # Generate and send a test event: vastde functions invoke --generate-event --url http://localhost:18080/ # Send a custom CloudEvent: vastde functions invoke --event cloudevent.yaml --url http://localhost:18080/ e8e703e11c8157e913a2c281868061455c236b107a599cf5b01c33ed7c016a1d Function '127.0.0.1:5000/docs/thumbnailer' ran successfully. ``` The last hex string is the container ID; `docker logs <container ID>` shows the runtime's log, and `docker stop <container ID>` stops and removes the container. Without `--detach`, the runtime's log goes to your terminal and Ctrl+C stops it; run `invoke` from a second terminal. If the image isn't on your machine, `localrun` pulls it from the registry first. ## Send it an event `vastde functions invoke --event <file>` sends the event in a YAML file to the local function. Write the file in the shape of a real event. This one is an element event for a new object, modelled on one a 5.5 cluster delivered: ``` # A test element event shaped like the ones an element trigger delivers specversion: "1.0" id: "87982405058561" type: vastdata.com:Element.ElementCreated source: vastdata.com:docs-src-uploads.7d5e6227-d1bf-4bf9-90c6-084333be5a98 subject: vast-broker-docs-kafka.default time: "2026-10-08T12:10:19.407648+00:00" datacontenttype: application/json elementpath: docs-src/photos/cat.jpg elementsourcetype: vast:s3 data: Records: - eventVersion: "2.2" eventSource: vast:s3 eventName: ObjectCreated:Put eventTime: "2026-10-08T12:10:19.407648Z" s3: bucket: name: docs-src object: key: photos/cat.jpg size: 24 ``` `elementpath` (`<bucket>/<key>`) is what `event.bucket` and `event.object_key` are read from. Keys other than the standard CloudEvent fields (`id`, `source`, `specversion`, `type`, `subject`, `time`, `data`) become extension attributes, as on the cluster. ``` vastde functions invoke --event element-event.yaml --url http://localhost:18080/ ``` ``` Sending CloudEvent to http://localhost:18080/ Event ID: 87982405058561 Event Type: vastdata.com:Element.ElementCreated CloudEvent sent successfully (Status: 204) ``` Status 204 means the handler returned without raising. If it raises, `invoke` prints `Failed to send CloudEvent (Status: 500)` and the exception's message. The handler's log lines are in the container's log: ``` docker logs ``` ``` … 2026-10-08 12:24:10 [INFO] [trace_id=-] START EventId: 87982405058561, TraceId: -, EventType: vastdata.com:Element.ElementCreated, EventSource: vastdata.com:docs-src-uploads.7d5e6227-d1bf-4bf9-90c6-084333be5a98, Timestamp: 1791462250.3285668 … 2026-10-08 12:24:10 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] new object: bucket=docs-src key=photos/cat.jpg 2026-10-08 12:24:10 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] Function returned: None in timestamp 2026-10-08 12:24:10.329816 with duration 0.49614906311035156 milliseconds … 2026-10-08 12:24:10 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] No sink URL configured 2026-10-08 12:24:10 [INFO] [trace_id=-] END EventId: 87982405058561, TraceId: -, Duration: 0.0017s, Response: , Timestamp: 1791462250.330168 2026-10-08 12:24:10 [INFO] [trace_id=-] 172.17.0.1:54444 - "POST / HTTP/1.1" 204 ``` Locally, `docker logs` prints most lines two or three times; on the cluster each appears once. **Don't rely on `--generate-event`.** `vastde functions invoke --generate-event` sends a built-in sample event. It's useful to check that the function starts and answers, but it isn't shaped like a real event: it has no `elementpath`, so `event.bucket` and `event.object_key` are `None`, its type and subject don't match what the cluster sends, and `event.get_data()` returns the whole event. See [test events locally](/develop/docs/dev-dataengine-reference-events#test-events-locally). Keep event files like the one above in your project, next to the code, and use them as test fixtures. ## Pass environment variables and secrets `--config` gives the local run environment variables (`envs`) and secrets (`secrets`, by secret name and key), in place of what the pipeline would attach on the cluster: ``` envs: STAGE: local THUMBNAIL_SIZE: "64" secrets: thumbnail-api: api-token: "" ``` Keep this file out of Git. Run the function **in the foreground** with it: ``` vastde functions localrun "$REGISTRY/docs/envcheck" --image-tag v1 --port 18081 --config localrun-config.yaml ``` ``` Running function '127.0.0.1:5000/docs/envcheck:v1' as Docker container on port 18081 Loaded configuration from: localrun-config.yaml Function is starting on port 18081... Once ready, you can invoke it: In another terminal, run: # Generate and send a test event: vastde functions invoke --generate-event --url http://localhost:18081/ # Send a custom CloudEvent: vastde functions invoke --event cloudevent.yaml --url http://localhost:18081/ Press Ctrl+C to stop the function 2026-10-08 13:06:32 [INFO] [trace_id=-] VAST DataEngine Runtime, Version: 0.2.0 … 2026-10-08 13:06:32 [INFO] [trace_id=-] Handler is now ready to accept requests 2026-10-08 13:06:32 [INFO] [trace_id=-] Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) 2026-10-08 13:06:36 [INFO] [trace_id=-] START EventId: 87982405058561, TraceId: -, EventType: vastdata.com:Element.ElementCreated, EventSource: vastdata.com:docs-src-uploads.7d5e6227-d1bf-4bf9-90c6-084333be5a98, Timestamp: 1791464796.605323 2026-10-08 13:06:36 [INFO] [trace_id=-] Function invoked in timestamp 2026-10-08 13:06:36.605654. Event batching mode: False 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] Starting running function in timestamp 2026-10-08 13:06:36.606112 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env STAGE='local' 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env THUMBNAIL_SIZE='64' 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env LOG_PREFIX=None 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] secrets: {'thumbnail-api': ['api-token']} 2026-10-08 13:06:36 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] api-token is 48 characters long … Received signal interrupt, stopping container... … Container stopped gracefully ``` The event came from `vastde functions invoke --event element-event.yaml --url http://localhost:18081/` in a second terminal, which printed `CloudEvent sent successfully (Status: 204)`. The handler is the `envcheck` function from [write a function](/develop/docs/dev-dataengine-guides-write-a-function#environment-variables-and-secrets). **Known issue in vastde v5.5.0-sp2: `--detach` drops the secrets.** With `--detach --config`, the container starts with the environment variables but with no secrets, because the CLI removes its temporary copy of them when it exits. A handler that reads a secret then fails: ``` vastde functions localrun "$REGISTRY/docs/envcheck" --image-tag v1 --port 18081 --config localrun-config.yaml --detach ``` ``` Running function '127.0.0.1:5000/docs/envcheck:v1' as Docker container on port 18081 Loaded configuration from: localrun-config.yaml Function is running in detached mode on port 18081! # Generate and send a test event: vastde functions invoke --generate-event --url http://localhost:18081/ # Send a custom CloudEvent: vastde functions invoke --event cloudevent.yaml --url http://localhost:18081/ 732e93c808a5e5102a8c4e253e16c1201eef2c23cc3d1ee9e85dc5ca5fb901ea Function '127.0.0.1:5000/docs/envcheck' ran successfully. ``` ``` vastde functions invoke --event element-event.yaml --url http://localhost:18081/ ``` ``` Sending CloudEvent to http://localhost:18081/ Event ID: 87982405058561 Event Type: vastdata.com:Element.ElementCreated Failed to send CloudEvent (Status: 500) Error Response: 'thumbnail-api' HTTP request failed with status 500 ``` ``` docker logs 2>&1 | grep -E '^20.*(env |secrets|api-token)' ``` ``` 2026-10-08 13:05:19 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env STAGE='local' 2026-10-08 13:05:19 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env THUMBNAIL_SIZE='64' 2026-10-08 13:05:19 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] env LOG_PREFIX=None 2026-10-08 13:05:19 [INFO] [trace_id=00005005-0000-0001-0000-000000000000] secrets: {} ``` So when you test with secrets, run `localrun` in the foreground. ## Catch start-up errors Errors when the handler file is imported (a missing package, a syntax error, a failing import at the top of the file) don't show up in the build. On the cluster, they leave the pipeline `InProgress` for minutes with nothing in `vastde logs` (see [write a function](/develop/docs/dev-dataengine-guides-write-a-function#dependencies)). Locally you see them at once, because the container exits with the traceback: ``` vastde functions localrun "$REGISTRY/docs/badimport" --image-tag v1 --port 18082 ``` ``` Running function '127.0.0.1:5000/docs/badimport:v1' as Docker container on port 18082 Function is starting on port 18082... Once ready, you can invoke it: In another terminal, run: # Generate and send a test event: vastde functions invoke --generate-event --url http://localhost:18082/ # Send a custom CloudEvent: vastde functions invoke --event cloudevent.yaml --url http://localhost:18082/ Press Ctrl+C to stop the function 2026-10-08 13:20:56 [INFO] [trace_id=-] VAST DataEngine Runtime, Version: 0.2.0 2026-10-08 13:20:56 [INFO] [trace_id=-] Starting VAST DataEngine Runtime 2026-10-08 13:20:56 [INFO] [trace_id=-] App Path: /workspace 2026-10-08 13:20:56 [INFO] [trace_id=-] Handler: main.py 2026-10-08 13:20:56 [INFO] [trace_id=-] Loading handlers from /workspace Container error output: Unable to find image '127.0.0.1:5000/docs/badimport:v1' locally … Status: Downloaded newer image for 127.0.0.1:5000/docs/badimport:v1 Traceback (most recent call last): … File "/workspace/main.py", line 1, in from PIL import Image # Pillow is not listed in requirements.txt ^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'PIL' docker container failed: exit status 1 ``` Run every new image locally once, with at least one event file, before you create or update the function. ## What a local run doesn't do A local run is the same image and the same runtime as on the cluster, but nothing around it: - **No downstream events.** There's no link, so the return value goes nowhere: the runtime logs `No sink URL configured`. Test a chain by sending the upstream function's return value, as the data of a function event file, to the downstream function. - **No triggers, retries or dead-letter topic.** Events arrive only when you send them, once. A handler that raises returns a 500 to `invoke`, and that's all. - **Logs stay local.** They go to the container's output, not to DataEngine: `vastde logs`, `vastde traces` and metrics show nothing from a local run. - **No cluster configuration.** Environment variables and secrets come only from `--config`, not from the pipeline file. Resources, scaling and timeouts don't apply. - **Event data may differ.** `invoke` sends the event file in a way that can make `event.get_data()` return the whole event rather than only `data`. Read objects with `event.bucket` and `event.object_key`, which behave the same in both places, and check code that reads `get_data()` against a deployed pipeline. ## See also - [Write a function](/develop/docs/dev-dataengine-guides-write-a-function) - [Publish functions](/develop/docs/dev-dataengine-guides-publish-functions): register the image and release new versions. - [Events](/develop/docs/dev-dataengine-reference-events): event shapes to copy into test files. - [Runtime SDK reference](/develop/docs/dev-dataengine-reference-runtime-sdk)