Security

Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.

Your platform team controls access to VAST DataBase with identity policies, and can add row and column security rules to them. This page explains how those policies affect what you see and what you can do from the SDK, Trino or Spark. It doesn't explain how to write them; that's an admin task, covered in the KB.

Who the cluster thinks you are

Every request is signed with an S3 access key. The cluster checks the request against the identity policies attached to that key's user (and its groups), and any bucket policy on the database. Effects to keep in mind:

Policies decide whether a request runs; row and column security decides what it returns.

• The SDK acts as the user who owns the key you pass to vastdb.connect().
• Trino and Spark act as the user who owns the key in the catalog or connector configuration, not as the person running the query, unless end-user impersonation is set up (see Impersonation). Everyone who uses that catalog gets the same access.
• Policies can deny as well as allow. A statement with "Effect": "Deny" blocks the listed actions even if another statement allows them (Identity and bucket policy reference).
• Policies can have conditions. For example, aws:SourceIp limits a statement to requests from certain client addresses. A key that works from one host can then be refused from another. If that happens, ask whether the policy has conditions.

Which actions each SDK call needs is on setup prerequisites. That page is the reference; this one doesn't repeat it.

What a refused request looks like

When the policy doesn't allow an action, the cluster answers with HTTP 403 and the SDK raises vastdb.errors.Forbidden at the call that needed the action. Nothing is retried: the SDK sends the refused request once. See errors.

Some refusals show up earlier or in a different place than you'd expect:

• Opening a database. tx.bucket(name) needs s3:ListBucket. Without it you get Forbidden with no error code, even if the policy allows every Tabular action.
• Opening a table. schema.table(name) loads the table's columns and statistics, so it needs s3:TabularListColumns and s3:TabularGetTableStats even if you only want to insert. Without s3:TabularListColumns, schema.table() fails on a request whose url ends in ?column.
• Updates and deletes. update() and delete() usually follow a select() that finds the rows, so they also need s3:TabularQueryData.
• Large inserts. If a batch doesn't fit in one request, the SDK inserts some columns first and writes the rest with update requests, so it needs s3:TabularUpdateRows as well as s3:TabularInsertRows. Without it, the insert fails on the update request (method PUT, url ending in ?rows), and the rows it already inserted are rolled back.
• Deleting partitions. delete_partitions() on a partitioned table needs s3:TabularDeletePartitions; s3:TabularDeleteRows isn't enough.
• VAST Query Engine. Without s3:QueryEngine*, every Query Engine statement on the table fails when it's planned, with Binder Error: [req=<id>]: permission checking for table: <bucket>/<schema>/<table> failed - AccessDenied., even if the SDK can read the table with the same key.

A failed call inside a transaction raises out of the with block, and the transaction rolls back. Earlier calls in the same transaction are undone too: an insert into one table that succeeded before a refused insert into another left no rows behind.

To test a key's access, run the operation you need and catch the error:

import vastdb
from vastdb import errors

session = vastdb.connect()
try:
    with session.transaction() as tx:
        table = tx.bucket("my-db").schema("sales").table("orders")
        table.select(columns=["id"], limit_rows=1).read_all()
    print("query allowed")
except errors.Forbidden as e:
    print("refused:", e.code, e.method, e.url)

Row and column security

On top of allowing or denying actions, an identity policy can hold row and column security statements. They change what a query returns, or which clients may read the table at all. The rules are (see Row and column security):

What this means for you:

• Two people can run the same query and get different results, in row counts, values and even column lists. A count that looks wrong may be a row filter. Ask your platform team which rules apply to your user before you debug the data.
• Masked values are not the data. Don't write masked results back to another table, or join on masked columns, and expect the real values.
• Rules apply to top-level columns only, not to fields nested inside a struct.
• Row filters and column masks are applied by the query engine. The database returns the filters and masks to apply, and the engine applies them. Masks use Trino SQL syntax (Managing identity policies).
• Engine restriction. If a policy defines row filters or column masks for you on a table, you can only query that table with one of the engines named in the policy's QueryEngine field (for example Trino). The Python SDK and the VAST Query Engine can't read such a table at all, even with "QueryEngine": "*"; see below.
• Column allow and deny also apply to inserts and renames, not only queries. An insert that includes a hidden column is refused.

What the Python SDK and the VAST Query Engine do

Column allow and deny, and the row filter by S3 path, are enforced by the cluster, so they work from every client. Row filters and column masks need an engine that applies them, and the Python SDK and the Query Engine refuse a table that has them for your user:

So read a table with row filters or column masks through Trino, and keep the Python SDK and the Query Engine for tables where your user has none. The 0x… values differ with the engines the policy names.

Which engines support it

• Trino applies row filters, column masks and column allow and deny, with impersonation set up in the catalog. See Trino.
• Spark: row filtering and column masking are supported for Trino only (a known limitation in 5.5), and impersonation isn't supported for Spark. Rely on row filters and column masks only with Trino. See Spark.
• The Python SDK and the VAST Query Engine honour column allow and deny and the row filter by S3 path, and refuse tables with row filters or column masks (above).

The vastdb_s3_path_auth column

Path-based row filtering reads a reserved column called vastdb_s3_path_auth. Each cell holds a comma-separated list of S3 paths, each in double quotes, such as "my_bucket/path1","my_bucket/path2". If you design a table for this:

• Add the column when you create the table. add_column() refuses it on an existing table (Cannot add column=vastdb_s3_path_auth, it can be added only during create), and it can't be dropped or renamed later.
• Every value must be a list of quoted paths. A value without the quotes, or a null, is refused with BadRequest (InvalidArgument).
• A row is returned only if the reader may read every path in its cell. With s3:GetObject on my_bucket/path1 only, a row holding "my_bucket/path1" was returned and a row holding "my_bucket/path1","my_bucket/path2" wasn't, from both the Python SDK and the Query Engine.

Impersonation

With impersonation, an engine or driver that holds one service key runs each request as the actual end user. The cluster then applies that person's identity policy, including row and column security.

In the SDK, end_user only applies to operations that go through the ADBC driver, such as vector_search(). Ordinary select(), insert(), update() and delete() calls still run as the key's owner: with end_user set to a user whose policy hides a column, select() still returned it.

Over ADBC, an impersonated query gets the end user's rules: a column denied to the end user isn't returned, and a table with a row filter for the end user is refused as described above. What you see when it isn't set up:

• The service key's user lacks s3:EndUserImpersonation: user login_name=<service user> … action=EndUserImpersonation is not allowed (end_user=<name>).
• The end user isn't a VAST user: login_name=<name> was not found.

Use these exact action names: s3:EndUserImpersonation and s3:TabularGetRowColumnSecurity. Similar names, such as s3:GetRowColumnSecurity or s3:TabularEndUserImpersonation, grant nothing. A policy that contains an action name the cluster doesn't recognise is still accepted; that action just grants nothing. s3:Tabular* covers s3:TabularGetRowColumnSecurity but not s3:EndUserImpersonation. Impersonation works only for Trino and the VAST Query Engine (ADBC).

For Trino, impersonation also needs three flags in the catalog file. Without them, row filters aren't applied, although denied columns are still removed. See Trino.

Protect the Trino server log. Trino connector 478 writes the catalog's secret access key to the Trino server log in plain text, in one INFO line per catalog when the catalog loads, although the configuration summary that follows shows it as [REDACTED]. Failed requests are also logged with their headers, which include the access key ID. Until this changes, restrict who can read the Trino server log, and where it's shipped, as you would the catalog file.

If your platform team uses protocol auditing, note that for impersonated access the audit log records only the original user's UID, not their login name or SID (a known limitation in 5.5).

What to ask your platform team

• Which identity policies apply to your key's user, and whether any statements deny actions or have conditions such as source IP.
• Whether any row and column security rules apply to your user, on which tables, and for which engines.
• For Trino: whether impersonation is set up, so rules apply to each person instead of to the catalog's user.
• For ADBC impersonation: s3:EndUserImpersonation for the service key.
• For the VAST Query Engine: s3:QueryEngine*.

Admin task: send to your platform team. See Managing permissions for accessing VAST tabular databases, Managing identity policies and Row and column security.

See also

• Setup prerequisites: the actions each SDK call needs, and example policies.
• Errors: reading a Forbidden error.
• Trino and Spark: row and column security from each engine.