---
title: "Tags and metadata"
slug: "dev-catalog-guides-tags-and-metadata"
updated: 2026-10-28T02:15:03Z
published: 2026-10-28T02:15:03Z
canonical: "docs.vastdata.com/dev-catalog-guides-tags-and-metadata"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tags and metadata

*Applies to VAST Cluster 5.5.*

If you classify data with S3 object tags or user metadata (`x-amz-meta-*` headers), the Catalog lets you find objects by them across a whole namespace: for example, every object whose `processed` tag is `false`. This guide shows the two ways: the `user_tags` and `user_metadata` map columns, which every Catalog has, and user-defined columns, which your platform team adds for the keys you use most.

Each code block is complete on its own. They read your connection settings from the environment, as in the [quickstart](/develop/docs/dev-catalog-quickstart#before-you-start), and use `CATALOG_PATH` as the subtree to search.

## Where tags and metadata are in the Catalog

| What | Column | Type |
| --- | --- | --- |
| All of an object's S3 tags | `user_tags` | map of key to value |
| How many tags it has | `user_tags_count` | integer |
| All of its S3 user metadata (`x-amz-meta-*`) | `user_metadata` | map of key to value |
| NFSv4.2 extended attributes | `nfs4_xattrs` | map |
| A tag you chose to index | `tag_<key>` | user-defined column |
| A metadata key you chose to index | `metadata_<key>` | user-defined column |
| An NFSv4.2 extended attribute you chose to index | `nfs4_xattr_<key>` | user-defined column |

You set tags and metadata with any S3 client: tags with `PutObjectTagging` (or on upload), metadata as `x-amz-meta-<key>` headers on `PutObject`. They show up in the Catalog after its next snapshot.

## Find objects by a tag

The Python SDK can't push down a filter on one key inside a map, so do it in two steps: filter on the cluster to objects that have tags at all (`user_tags_count > 0`), then look up the key in your client.

This example needs objects under your path tagged `processed=false`:

```
import os

import pyarrow.compute as pc
import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

session = vastdb.connect()
with session.transaction() as tx:
    rows = tx.catalog().select(
        columns=["parent_path", "name", "user_tags"],
        predicate=(_.search_path == path) & (_.element_type == "FILE") & (_.user_tags_count > 0),
    ).read_all()

processed = pc.map_lookup(rows["user_tags"], "processed", "first")
todo = rows.filter(pc.equal(processed, "false"))

print(rows.num_rows, "tagged objects,", todo.num_rows, "with processed=false")
for row in todo.slice(0, 10).to_pylist():
    print(row["parent_path"] + row["name"], dict(row["user_tags"]))
```

```
9 tagged objects, 4 with processed=false
/cat-data/projects/apollo/part-008.parquet {'processed': 'false', 'project': 'apollo'}
/cat-data/projects/apollo/part-007.parquet {'processed': 'false', 'project': 'apollo'}
/cat-data/projects/apollo/part-006.parquet {'processed': 'false', 'project': 'apollo'}
/cat-data/projects/apollo/model.bin {'processed': 'false', 'project': 'apollo'}
```

`map_lookup` returns null for objects that don't have the key, and the comparison leaves them out.

## Find objects by user metadata

User metadata works the same way, with `user_metadata`. The map has no count column, and a filter on a map column can't run on the cluster (`isnull()` on it raises `NotImplementedError`), so fetch the column and drop the objects without metadata in your client. On a large namespace, narrow the query with `search_path`, or ask for the key to be indexed (next section). This example needs an object under your path uploaded with the header `x-amz-meta-project: apollo`:

```
import os

import pyarrow.compute as pc
import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

session = vastdb.connect()
with session.transaction() as tx:
    rows = tx.catalog().select(
        columns=["parent_path", "name", "user_metadata"],
        predicate=(_.search_path == path) & (_.element_type == "FILE"),
    ).read_all()

# A filter on a map column can't run on the cluster, so drop the objects without metadata here.
rows = rows.filter(pc.is_valid(rows["user_metadata"]))
for row in rows.slice(0, 10).to_pylist():
    print(row["parent_path"] + row["name"], dict(row["user_metadata"]))

project = pc.map_lookup(rows["user_metadata"], "project", "first")
print(rows.filter(pc.equal(project, "apollo")).num_rows, "objects with project=apollo")
```

```
/cat-data/reports/q3-summary.pdf {'department': 'finance', 'project': 'apollo'}
/cat-data/projects/gemini/sales-4.csv {'project': 'gemini'}
/cat-data/projects/gemini/sales-3.csv {'project': 'gemini'}
/cat-data/projects/gemini/sales-1.csv {'project': 'gemini'}
/cat-data/projects/gemini/sales-2.csv {'project': 'gemini'}
1 objects with project=apollo
```

The first lines print each object's metadata as it's stored, so you can see the exact keys. The keys are stored without the `x-amz-meta-` prefix.

## Filter on a user-defined column

When you filter on the same key often, or on a large namespace, ask your platform team to index it as a user-defined attribute. **Admin task: send to your platform team:** [Configuring VAST Catalog](/docs-5-5/docs/configuring-vast-catalog), "User defined attributes". Tell them the type (tag, metadata or NFSv4 xattr) and the key.

From the next snapshot on, the Catalog has a column for that key, such as `tag_processed`, and a filter on it pushes down like any other column. The tag is still in `user_tags` too.

This example needs a user-defined tag attribute `processed`:

```
import os

import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

session = vastdb.connect()
with session.transaction() as tx:
    rows = tx.catalog().select(
        columns=["parent_path", "name", "tag_processed"],
        predicate=(_.search_path == path) & (_.tag_processed == "false"),
    ).read_all()

print(rows.num_rows, "objects with processed=false")
```

```
4 objects with processed=false
```

To see which user-defined columns your Catalog has, list its columns and look for the `tag_`, `metadata_` and `nfs4_xattr_` prefixes:

```
import vastdb

session = vastdb.connect()
with session.transaction() as tx:
    fields = tx.catalog().columns()

for field in fields:
    if field.name.startswith(("tag_", "metadata_", "nfs4_xattr_")):
        print(field.name, field.type)
```

```
metadata_project string
tag_processed string
```

## In SQL

With Trino, filter on a map key with `element_at`, for example `WHERE element_at(user_tags, 'processed') = 'false'`. Trino evaluates that itself; whether the connector pushes it down to the cluster isn't documented, so keep a `search_path` filter in the query. See [Trino and Spark](/develop/docs/dev-catalog-integrations-trino-and-spark).

## See also

- [Concepts: indexed and user-defined columns](/develop/docs/dev-catalog-concepts-index#indexed-and-user-defined-columns).
- [Catalog schema](/develop/docs/dev-catalog-reference-schema).
- [Find files](/develop/docs/dev-catalog-guides-find-files).