Developer tenant

Applies to VAST Cluster 5.5.

Introduction

Without a developer tenant, getting started on VAST means a request to your platform team for each service: a database, a bucket, a Kafka view, a user, keys and policies. Each one is an admin task, and each new idea means another request. A developer tenant replaces them with one request.

We strongly recommend a developer tenant on every VAST cluster your developers build on: at least one per cluster, and one for each team if several teams of developers use the cluster. It's the quickest way to run every quickstart and try your own ideas, it keeps your experiments away from shared and production data, and it saves your platform team a ticket for each service. With DataEngine in it, you can build complete pipelines in one place: an event, a function that handles it, and the tables and objects it writes.

A developer tenant is your team's own tenant on a VAST cluster. It isn't a separate system or a VM: it's part of the cluster you'll use for real work, with its own users, keys, buckets and limits, and a tenant admin login for you, or for whoever looks after it in your team. Your platform team sets it up once. After that you create your own users (one for each developer), keys, databases, buckets, Kafka views and topics, without a ticket for each one. Everything you create stays in your tenant, separate from other teams' developer tenants: your keys work only there, and the VAST Catalog shows you only your tenant's files and objects.

What you can run in it. The VAST DataBase, VAST Event Broker and VAST DataStore quickstarts run against a developer tenant as they are. VAST Vector Search and SQL through the VAST Query Engine need one more item from your platform team, a Query Engine pool, and the VAST Catalog needs to be switched on for the cluster. VAST DataEngine needs a compute cluster and a container registry: ask for them with your developer tenant (see add VAST DataEngine). One compute cluster can serve every developer tenant on the cluster. If your platform team can't spare the compute, everything else in the developer tenant works without it.

What it's for. Learning the services, trying an idea, building a prototype, and testing your code before it goes anywhere else. It isn't for production data: the setup below gives your user every action on the tenant's buckets, which is fine for a developer tenant and too broad for production.

How it works.

1. You send your platform team the request below: a tenant, a tenant admin login, a virtual IP pool and network access, the compute cluster and registry for DataEngine, and any optional items you need.
2. You set up your developer tenant: hand it to an AI coding assistant, or run a short Python script yourself. Either way you get a user and key pair, a database, a Kafka view and a bucket, with the settings saved to a file.
3. You run the quickstarts with those settings, and build from there.
4. You add VAST DataEngine to your tenant once its compute cluster and registry are ready, and build pipelines.

Your platform team sets up once; then you and your code work on your own.

What to ask your platform team for

Admin task: send to your platform team. These items need a cluster admin. A tenant admin can't create virtual IP pools, Query Engine pools included, and can't see or change the Catalog settings.

Please set up a developer tenant for <team> on <cluster>:

1. A tenant, for example dev-<team>, with its own VAST (local) provider
   that tenant admins are allowed to manage.
2. A tenant admin login for me on that tenant: a role on the tenant with
   create, view, edit and delete on the Logical, Security and Database realms
   (Monitoring and Events too, if I should see metrics and alarms).
3. A virtual IP pool with the Protocols role, assigned to the tenant.
   Send me the pool name and the S3 endpoint URL (one of its virtual IPs,
   or its DNS name).
4. Network access from <my machine or subnet> to every virtual IP in the
   pool (S3 port, and 9092 for Kafka) and to the VMS address (HTTPS).
5. Only if I'll use the VAST Query Engine or VAST Vector Search: a virtual IP
   pool with the Query Engine role for the tenant, on CNodes that aren't in
   another Query Engine pool.
6. Only if I'll use the VAST Catalog: the Catalog enabled on the cluster.
7. For VAST DataEngine (recommended): my tenant attached to a
   compute cluster (a VAST compute cluster or an external Kubernetes
   cluster), and a container registry served over HTTPS with a valid CA
   certificate, linked to my tenant's namespace, with push credentials
   for me. See "Add VAST DataEngine" on the developer tenant page.

The procedures are in the KB: creating tenants, VAST providers, managing tenant administrators and administrative roles, virtual IP pools, the VAST Query Engine and configuring the VAST Catalog.

Why each item:

• The provider holds the users you'll create. The cluster's default provider is managed by cluster admins only, so a tenant admin can't add users to it.
• One Protocols pool is enough for S3, VAST DataBase and VAST Event Broker: the S3 endpoint and the Kafka bootstrap servers can be the same virtual IPs. A pool takes one Kafka view with unencrypted connections, so one Kafka view per developer tenant.
• The Query Engine pool needs CNodes of its own. A CNode can be in only one Query Engine pool, and creating a second pool on the same CNodes fails with INVALID_PARAM: CNODE_BELONGS_TO_OTHER_QUERY_ENGINE_VIP. Keys from your developer tenant don't work on another tenant's pool, so ask for one for your tenant.
• The Catalog is one switch for the whole cluster. A tenant admin can't see or change it. Once it's on, you see only your own tenant's files and objects; see the Catalog access page.
• DataEngine needs compute that only a cluster admin can provide, and that costs the cluster resources; see add VAST DataEngine. A tenant admin can't add compute clusters or registries.

Set up your developer tenant

You do the rest yourself, as the tenant admin. You don't need to know how the cluster is organized: an AI coding assistant can run the steps for you, or you can run them yourself.

With an AI coding assistant

You can hand the setup to an AI coding assistant, such as Claude Code:

1. Set the environment variables from before you start, including your tenant admin login, in the shell the assistant works in. Don't paste the password or keys into the chat.
2. Give the assistant this page, and tell it that your cluster's API reference is at https://<VMS address>/docs/.
3. Ask it to run the steps below in one Python session, then the quickstart for each service you want to try.
4. Review each command before you allow it. The assistant acts as your tenant admin: that login is limited to your tenant, but it can still delete what's in it.

An assistant given only this page, the API reference and a tenant admin login can set up a new developer tenant and run the VAST DataBase, VAST Event Broker and VAST DataStore quickstarts in about ten minutes.

Don't give an assistant a tenant admin login for anything other than a developer tenant.

The VAST admin MCP server, vast-admin-mcp, gives an assistant a read-only view of your developer tenant's views, policies and keys, but it can't do the setup: see automation tools.

Step by step

These are the steps an assistant runs, and you can run them yourself just as well. They use vastpy; the VMS REST API and the VAST Web UI work too. They create:

<name> is a short name for your developer tenant, such as your team's name. Bucket names are unique across the whole cluster, not just your tenant, so a name another tenant already uses fails with 409 "View parameters conflict with another view: Bucket already exists".

Run the blocks in order in one Python session, such as a notebook or one interactive python prompt: later blocks use variables from earlier ones, and the secret key exists only in memory until save your settings writes it to a file.

One Protocols pool serves S3, VAST DataBase and Kafka for the developer tenant.

The identity policy allows everything on the tenant's buckets, which is fine for a developer tenant and too much for production. For production policies, see the VAST DataBase setup prerequisites.

Before you start

You need Python 3.10 or later, the details from your platform team, and the tenant admin password:

pip install "vastpy==0.3.22" "boto3==1.40.0" "vastdb==2.1.0"
export VMS_ADDRESS="vms.example.com"                     # the VMS address, no https://
export VAST_TENANT="dev-<team>"                      # your tenant
export VMS_USER="<tenant admin user>"
export VMS_PASSWORD="<tenant admin password>"
export TENANT_POOL="<virtual IP pool name>"
export TENANT_ENDPOINT="http://<virtual IP>"             # the S3 endpoint URL your platform team sent
export TENANT_NAME="<team>"                             # prefix for your bucket names

vastpy doesn't check the VMS certificate unless you give it a CA file (cert_file=); see automation tools.

Connect

import ipaddress
import json
import os

import boto3
from vastpy import VASTClient

vms = VASTClient(
    address=os.environ["VMS_ADDRESS"],
    tenant=os.environ["VAST_TENANT"],
    user=os.environ["VMS_USER"],
    password=os.environ["VMS_PASSWORD"],
)
tenant = vms.tenants.get(name=os.environ["VAST_TENANT"])[0]
pool = vms.vippools.get(name=os.environ["TENANT_POOL"])[0]
name = os.environ["TENANT_NAME"]
print("tenant:", tenant["name"])
print("pool:", pool["name"], pool["role"], pool["start_ip"], "-", pool["end_ip"])

tenant: sb2-tenant
pool: sb2-proto PROTOCOLS 15.0.0.72 - 15.0.0.73

Create a view policy

view_policy = vms.viewpolicies.post(
    name="sb-policy",
    tenant_id=tenant["id"],
    flavor="S3_NATIVE",
    nfs_read_write=["*"],
    s3_read_write=["*"],
    smb_read_write=["*"],
    nfs_root_squash=["*"],
)
view_policy = vms.viewpolicies[view_policy["id"]].patch(is_s3_default_policy=True)
print(view_policy["name"], view_policy["flavor"], "| for new S3 buckets:", view_policy["is_s3_default_policy"])

sb-policy S3_NATIVE | for new S3 buckets: True

is_s3_default_policy is "Use for new S3 buckets" in the Web UI. Without such a policy on the tenant, S3 CreateBucket fails with AccessDenied.

Create a user and a key pair

identity_policy = vms.s3policies.post(
    name="sb-app-policy",
    tenant_id=tenant["id"],
    policy=json.dumps({
        "Version": "2012-10-17",
        "Statement": [{"Effect": "Allow", "Action": ["s3:*"], "Resource": ["*"]}],
    }),
)
user = vms.users.post(
    name="sb-app",
    local_provider_id=tenant["local_provider_id"],
    allow_create_bucket=True,
    allow_delete_bucket=True,
    s3_policies_ids=[identity_policy["id"]],
)
key = vms.users[user["id"]].access_keys.post(tenant_id=tenant["id"])
print("user:", user["name"], "| policies:", [p["name"] for p in user["s3_policies"]])
print("key pair created:", bool(key["access_key"] and key["secret_key"]))

user: sb-app | policies: ['sb-app-policy']
key pair created: True

The secret key is returned only once, here. The next step writes it to a file; don't print it.

Create a database and a Kafka view

database = vms.views.post(
    name=f"{name}-db", path=f"/{name}-db", create_dir=True, bucket=f"{name}-db", bucket_owner="sb-app",
    policy_id=view_policy["id"], protocols=["S3", "DATABASE"], tenant_id=tenant["id"],
)
broker = vms.views.post(
    name=f"{name}-events", path=f"/{name}-events", create_dir=True, bucket=f"{name}-events", bucket_owner="sb-app",
    policy_id=view_policy["id"], protocols=["S3", "DATABASE", "KAFKA"],
    kafka_vip_pools=[pool["id"]], is_kafka_unencrypted_conn_allowed=True, tenant_id=tenant["id"],
)
for view in (database, broker):
    print(view["path"], view["bucket"], view["protocols"])

/sb2-db sb2-db ['S3', 'DATABASE']
/sb2-events sb2-events ['S3', 'DATABASE', 'KAFKA']

The Kafka view takes unencrypted connections on port 9092, with no authentication. For TLS, SASL/PLAIN and authorization, see the Event Broker security page.

Save your settings

Write the endpoint, the key pair and the names to devtenant.env, readable only by you, with the variable names the quickstarts use:

first = ipaddress.ip_address(pool["start_ip"])
last = ipaddress.ip_address(pool["end_ip"])
vips = [str(first + i) for i in range(int(last) - int(first) + 1)]
settings = {
    "AWS_S3_ENDPOINT_URL": os.environ["TENANT_ENDPOINT"],
    "AWS_ACCESS_KEY_ID": key["access_key"],
    "AWS_SECRET_ACCESS_KEY": key["secret_key"],
    "VASTDB_BUCKET": f"{name}-db",
    "EVENTS_BUCKET": f"{name}-events",
    "KAFKA_BOOTSTRAP": ",".join(f"{ip}:9092" for ip in vips[:3]),
    "DATASTORE_BUCKET": f"{name}-files",
}
fd = os.open("devtenant.env", os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(fd, "w") as f:
    f.writelines(f"{name}={value}\n" for name, value in settings.items())
print("wrote devtenant.env:", ", ".join(settings))

wrote devtenant.env: AWS_S3_ENDPOINT_URL, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, VASTDB_BUCKET, EVENTS_BUCKET, KAFKA_BOOTSTRAP, DATASTORE_BUCKET

Keep devtenant.env out of Git. The bootstrap servers are up to three virtual IPs from the pool (two are enough); every virtual IP in the pool must be reachable from your client.

If a step fails partway, the objects created before it stay. Running the whole page again then fails on their names: delete them first, as in clean up, or carry on from the step that failed in the same session. If you lose the secret key before it's saved, delete that key pair with the VMS REST API (DELETE /api/latest/users/<user id>/access_keys/, with the access_key in the body) and create a new one.

Create a bucket as your user

Your user can create buckets with the S3 API:

s3 = boto3.client(
    "s3",
    endpoint_url=settings["AWS_S3_ENDPOINT_URL"],
    aws_access_key_id=settings["AWS_ACCESS_KEY_ID"],
    aws_secret_access_key=settings["AWS_SECRET_ACCESS_KEY"],
)
s3.create_bucket(Bucket=settings["DATASTORE_BUCKET"])
print(sorted(b["Name"] for b in s3.list_buckets()["Buckets"]))

['sb2-db', 'sb2-events', 'sb2-files']

Run the quickstarts

Load the settings into your shell, then follow each quickstart from its "Connect" or "Set up" step. You can skip its "Before you start" list: the developer tenant has everything it asks for.

set -a; . ./devtenant.env; set +a

Add VAST DataEngine

We recommend DataEngine in every developer tenant, so that you can build and test pipelines from the start. DataEngine runs your functions on a compute cluster, so it's the one part of a developer tenant that costs the cluster resources. The table below shows what each choice costs, so your platform team can plan for it. If they can't provide it, the rest of the developer tenant works without it.

What your platform team provides. One compute cluster and one registry can serve every developer tenant on the cluster: each tenant gets its own namespace.

What you do, as the tenant admin. Enable DataEngine on your tenant, with <name>-events as its event broker and two topics you create there first: the default topic and the dead-letter topic. Choose their partition counts with care, because a topic's partition count can't be changed later. Then make your users' group the tenant's DataEngine application users group, and turn on the DataEngine role and identity policy for it. You can do all of this without a compute cluster, but pipelines can't be deployed until the compute cluster and the registry are linked. The steps are in the KB's tenant guide.

What you install. Docker, the vastde CLI and a builder image, and push access to the registry. See your workstation.

Then follow the DataEngine setup prerequisites to check your setup, and run the DataEngine quickstart.

What you can do yourself

As the tenant admin of your developer tenant, you can create, change and delete these on your tenant, with the Web UI, the REST API or vastpy:

• Local users and groups, S3 key pairs, and identity policies.
• View policies, including the one for new S3 buckets.
• Views: databases (S3 and DATABASE), buckets and Kafka views on your tenant's pools.
• Event Broker topics (POST /api/latest/topics/ with database_name, or the Kafka Admin API as your user).
• DataEngine on the tenant, and which group's users are its application users. Pipelines deploy once your platform team has provided the compute cluster and registry (add VAST DataEngine).

Your user sb-app creates schemas and tables in <name>-db, topics in <name>-events and buckets with the S3 API, with no admin step.

You still need your platform team for: new virtual IP pools (a tenant admin gets HTTP 403), a Query Engine pool, the Catalog, compute clusters for DataEngine, and anything outside your tenant.

Clean up

To remove what this page created, delete the bucket's objects first and any schemas you made, then the objects in reverse order:

import time

import vastdb

files = settings["DATASTORE_BUCKET"]
for item in s3.list_objects_v2(Bucket=files).get("Contents", []):
    s3.delete_object(Bucket=files, Key=item["Key"])
s3.delete_bucket(Bucket=files)

session = vastdb.connect(
    endpoint=settings["AWS_S3_ENDPOINT_URL"],
    access=settings["AWS_ACCESS_KEY_ID"],
    secret=settings["AWS_SECRET_ACCESS_KEY"],
)
for bucket in (settings["VASTDB_BUCKET"], settings["EVENTS_BUCKET"]):
    with session.transaction() as tx:
        for schema in tx.bucket(bucket).schemas():
            for table in schema.tables():
                table.drop()
            schema.drop()

vms.views[broker["id"]].delete()
vms.views[database["id"]].delete()
vms.users[user["id"]].delete()
vms.s3policies[identity_policy["id"]].delete()
for _ in range(30):  # the bucket's own view goes a moment after DeleteBucket
    if not [v for v in vms.views.get() if v.get("policy_id") == view_policy["id"]]:
        break
    time.sleep(2)
vms.viewpolicies[view_policy["id"]].patch(is_s3_default_policy=False)
vms.viewpolicies[view_policy["id"]].delete()
os.remove("devtenant.env")
print("removed")

removed

A Kafka view always has a kafka_topics schema, and a view with schemas can't be deleted ("Deleting a view with created schemas or tables is not allowed"), so the loop drops them first. A view policy can't be deleted while it's the policy for new S3 buckets, or while a view still uses it: the view that S3 CreateBucket made for your bucket goes a moment after delete_bucket(), and until then deleting the policy fails with 409 "Cannot delete view policy that has referenced views", so the loop waits for it.

See also

• Get access: what each service needs, when your platform team sets it up for you.
• Automation tools: the REST API, vastpy, Terraform and the admin MCP server.
• Each service's CI/CD and automation page, for example VAST DataBase.