--- title: "Security" slug: "dev-dataengine-operations-security" description: "Learn about DataEngine user roles, identity policies, and managing secrets for secure application development in Kubernetes environments." tags: ["DataEngine"] updated: 2026-10-30T18:19:06Z published: 2026-10-30T18:19:06Z canonical: "docs.vastdata.com/dev-dataengine-operations-security" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Security *Applies to VAST Cluster 5.5.* Your platform team decides what you can do in DataEngine: your user type, and for application users, the identity policies on your user and groups. Your functions then need their own credentials, which you pass as Kubernetes secrets. This page covers both, plus registry credentials and the credentials the CLI keeps. ## Who can do what | User | What they can do in DataEngine | | --- | --- | | Cluster admin | Everything, on every tenant. Creates and manages VAST compute clusters. | | Tenant admin | Everything on their tenant, automatically: enabling DataEngine, linking compute clusters and container registries, creating source views, and all triggers, functions and pipelines. Source views need permission for the Logical realm, and DataEngine setup needs the Security realm. | | Application user | Only what their identity policies allow, on triggers, functions and pipelines. No tenant administration. | Most developers are application users. An application user belongs to a group, on a VAST, LDAP or Active Directory provider, that the tenant grants DataEngine access to. The group also needs a role on the tenant: without one, VMS refuses every request from its users with HTTP 403. Two more things before an application user can work: - **A new password.** A password set by your platform team is temporary. Until you set your own, VMS refuses every request from your user with HTTP 403 and the code `password_change_required`, so the CLI fails too. The VAST Web UI asks for a new password when you log in. - **An S3 access key pair** for your user. Without one, VMS refuses your DataEngine requests with HTTP 403 "You do not have permission to perform this action." ### What application users can't do Whatever their policies say, application users can't: - Enable or set up DataEngine on the tenant. - Link, change or remove compute clusters. - Link, change or remove container registries. - Create source views for element triggers. The first three fail with HTTP 403 and "App users are not allowed to perform this action". Creating a view fails with HTTP 403 "You do not have permission to perform this action.": source views are created by a tenant or cluster admin. Application users can still list compute clusters and registries, to find the names to use in their YAML. ## Identity-policy permissions Identity policies grant DataEngine actions on DataEngine resources: | Action | Allows | | --- | --- | | `dataengine:CreateTrigger` | Creating triggers | | `dataengine:GetTrigger` | Listing and reading triggers | | `dataengine:UpdateTrigger` | Changing triggers | | `dataengine:DeleteTrigger` | Deleting triggers | | `dataengine:CreateFunction` | Creating functions | | `dataengine:GetFunction` | Listing and reading functions | | `dataengine:UpdateFunction` | Changing functions (new revisions) | | `dataengine:DeleteFunction` | Deleting functions | | `dataengine:CreatePipeline` | Creating pipelines, including their function deployments | | `dataengine:GetPipeline` | Listing and reading pipelines | | `dataengine:UpdatePipeline` | Changing pipelines | | `dataengine:DeletePipeline` | Deleting pipelines | Resources are VAST resource names (VRNs): `vast:dataengine:triggers:<name>`, `vast:dataengine:functions:<name>` and `vast:dataengine:pipelines:<name>`, with `*` matching every name, for example `vast:dataengine:pipelines:*`. A `*` inside a name matches the rest of it: `vast:dataengine:triggers:media-*` matches the triggers whose names start with `media-`. ### The predefined policy When your platform team enables "Assign DataEngine identity policy to group" for the DataEngine group, its users get the `data-engine-<tenant>` policy. It grants: - `dataengine:CreateTrigger`, `dataengine:CreateFunction` and `dataengine:CreatePipeline` on all triggers, functions and pipelines. These also let a user view, update and delete the triggers, functions and pipelines **they created**. - Read access to DataEngine's own databases (the `dataengine-*` buckets): `s3:HeadBucket`, `s3:Tabular*Transaction`, `s3:TabularList*`, `s3:TabularGet*` and `s3:TabularQueryData`. So with the predefined policy alone, each user works on their own resources. Other users' triggers don't appear in your lists, and reading, changing or deleting another user's trigger, or reading their function, fails with HTTP 404, for example "Trigger with (guid=…) not found!". If a team shares pipelines, or a CI user deploys what developers created, ask for the `Get`, `Update` and `Delete` actions too. An example policy for a team that shares everything on the tenant. It keeps the predefined policy's database statement and grants every DataEngine action: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "DataEngineTablesAccess", "Effect": "Allow", "Action": [ "s3:HeadBucket", "s3:Tabular*Transaction", "s3:TabularList*", "s3:TabularGet*", "s3:TabularQueryData" ], "Resource": ["dataengine-*", "dataengine-*/*"] }, { "Sid": "DataEngineTeam", "Effect": "Allow", "Action": [ "dataengine:CreateTrigger", "dataengine:GetTrigger", "dataengine:UpdateTrigger", "dataengine:DeleteTrigger", "dataengine:CreateFunction", "dataengine:GetFunction", "dataengine:UpdateFunction", "dataengine:DeleteFunction", "dataengine:CreatePipeline", "dataengine:GetPipeline", "dataengine:UpdatePipeline", "dataengine:DeletePipeline" ], "Resource": [ "vast:dataengine:triggers:*", "vast:dataengine:functions:*", "vast:dataengine:pipelines:*" ] } ] } ``` With this policy, a user lists and changes the triggers that other users created. To limit a user to some resources, use names in the resources, for example `vast:dataengine:pipelines:media-*`. Identity policies cover DataEngine resources only. What your **functions** can read and write on the cluster depends on the S3 keys you give them; see [S3 keys for functions](/develop/docs/dev-dataengine-operations-security#s3-keys-for-functions). ## Secrets Use secrets for anything sensitive that a function needs: API keys, passwords, S3 keys, tokens. A DataEngine secret is a Kubernetes secret in the pipeline's namespace on the compute cluster, holding one or more key-value pairs. ### Create a secret Create or update a secret when you create or update a pipeline, with `--secret-file`: ``` # llm-creds.secret.yaml: keep this file out of Git name: llm-creds kubernetes_cluster_vrn: vast:dataengine:kubernetes-clusters: namespace: entries: - key: API_KEY value: "" - key: ENDPOINT value: "https://llm.example.com/v1" ``` ``` vastde pipelines update my-pipeline --config pipeline.yaml --secret-file llm-creds.secret.yaml ``` `--secret-file` can be repeated. The CLI creates or updates each secret before it sends the pipeline. If a secret with that name already exists in the namespace, its entries are replaced. ### Attach it Creating a secret doesn't attach it to anything. Name it in the [pipeline YAML](/develop/docs/dev-dataengine-reference-pipeline-yaml), either for every function in the pipeline or for one function deployment: ``` manifest: config: secrets: [shared-db-creds] # every function deployment function_deployments: - name: summarise function_vrn: vast:dataengine:functions:summarise config: secrets: [llm-creds] # this deployment only ``` - A secret can be attached at the pipeline level or to a function deployment, not both. - DataEngine checks that each attached secret exists when you deploy, not when you create or update the pipeline. The CLI only warns about a secret it can't find. ### Read it Each attached secret is mounted into the function's container, and `ctx.secrets` returns them as `{secret name: {key: value}}`. See the [Runtime SDK reference](/develop/docs/dev-dataengine-reference-runtime-sdk#secrets). The runtime reads the secrets once per instance and keeps them. After you change a secret, redeploy the pipeline so that new instances read the new values. ### Keep secrets secret - **Not in environment variables.** Environment variables are visible to anyone who can read the pipeline's configuration. - **Not in the image.** Anyone who can pull an image can read every file in it. - **Not in Git.** The pipeline YAML refers to secrets by name only, so it can live in Git. Keep `--secret-file` files out of it (for example with a `.gitignore` entry), or generate them in CI from your secrets store. - **Not in logs or return values.** Mask secrets before logging them. The runtime writes each handler's return value to the function's logs, and the return value travels to the next function as event data, so don't return a secret. ## Container registry credentials Two different things need access to your registry: - **Your machine or CI pushes images.** `vastde functions build --push` (and `docker push`) use your local Docker credentials, from `docker login`. Ask the registry's owner for push credentials, and use a CI-specific token where you can. - **The compute cluster pulls images.** This uses the registry's authentication method in DataEngine, set by your platform team when they link the registry: | Authentication method | How the compute cluster authenticates | | --- | --- | | User credentials | DataEngine stores a user name and password and creates the pull secret for you in the namespaces of the linked clusters. | | Kubernetes secret | Your platform team creates an image pull secret (type `kubernetes.io/dockerconfigjson`) in each namespace in advance, and names it. DataEngine checks that it exists. | | None | The registry allows anonymous pulls. | Registry links are an admin task: application users can't create or change them. The registry must use HTTPS with a certificate from a valid CA. Pull credentials only need read access: ask for a read-only account or token. ## S3 keys for functions DataEngine doesn't give your functions any access to the cluster's data. A function that reads an object when it's uploaded, writes results to a bucket, or queries a VAST DataBase table does so with the S3 access keys you give it. - **Use a separate user.** Ask your platform team for a user for your functions, with an identity policy that allows only the buckets and tables they use, rather than passing your own keys. For S3 objects that's the usual S3 actions; for VAST DataBase tables, the `s3:Tabular*` actions in the DataBase [setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites#permissions). - **Pass the keys as a secret**, and create the client in `init`. See the [Runtime SDK reference](/develop/docs/dev-dataengine-reference-runtime-sdk#s3client) and [pass the credentials](/develop/docs/dev-dataengine-integrations-vast-database-and-vector-search#pass-the-credentials). - **Pass the endpoint as an environment variable.** It isn't secret. Don't name it with a `VAST_` prefix, which is reserved; see [environment variables](/develop/docs/dev-dataengine-reference-runtime-sdk#environment-variables). - **Rotate** by creating a new key pair, updating the secret and redeploying, then deleting the old key pair. S3 keys are per tenant: a key works only on your tenant's endpoints. Your own key pair is for your use of DataEngine, such as browsing views, and for tools you run yourself. As an application user, you need it before DataEngine accepts your requests; see [who can do what](/develop/docs/dev-dataengine-operations-security#who-can-do-what). ## CLI credentials `vastde config init` saves your user name and password in `~/.vast/config.toml`, readable only by you. The CLI logs in with them on every command. On shared machines and in CI, set them as environment variables instead of writing the file: `VAST_VMS_URL`, `VAST_TENANT`, `VAST_USERNAME` and `VAST_PASSWORD` (and `VAST_BUILDER_IMAGE_URL` for builds). Take them from your CI system's secret store. Use a dedicated application user for CI, with only the actions it needs, so that you can rotate or revoke its password without affecting people. ## What to ask your platform team - An application user in the DataEngine group, with a role on the tenant, and an S3 key pair for that user. Set your own password before you start. - The `data-engine-<tenant>` policy, or a policy with the actions above. Say whether you share triggers, functions and pipelines with other users. - A separate user and key pair for your functions, with access to the buckets and tables they use. - A dedicated application user for CI, if you deploy from CI. - The container registry's name in DataEngine, how it authenticates pulls, and push credentials for your machine or CI. - Source views for your element triggers. The procedures are in the KB: [provisioning user access and permissions](/docs-5-5/docs/provisioning-user-access-and-permissions-for-dataengine), [configuring identity policies for DataEngine](/docs-5-5/docs/provisioning-user-access-and-permissions-for-dataengine#configuring-identity-policies-for-dataengine) and [managing container registries](/docs-5-5/docs/managing-container-registries). ## See also - [Setup prerequisites](/develop/docs/dev-dataengine-operations-setup-prerequisites) - [Runtime SDK reference](/develop/docs/dev-dataengine-reference-runtime-sdk): `ctx.secrets` and environment variables - [Pipeline YAML](/develop/docs/dev-dataengine-reference-pipeline-yaml): attaching secrets and environment variables - [VAST DataBase security](/develop/docs/dev-database-operations-security) and [VAST Event Broker security](/develop/docs/dev-event-broker-operations-security)