---
title: "Snapshots and change detection"
slug: "dev-catalog-guides-snapshots-and-change-detection"
updated: 2026-10-30T18:18:51Z
published: 2026-10-30T18:18:51Z
canonical: "docs.vastdata.com/dev-catalog-guides-snapshots-and-change-detection"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Snapshots and change detection

*Applies to VAST Cluster 5.5.*

The cluster keeps earlier copies of the Catalog table as Catalog snapshots. Comparing two of them tells you what changed in between: which files were deleted, which were added, and, with care, which were moved. This guide lists the snapshots, reads one, and compares two.

Each code block is complete on its own. They read your connection settings from the environment, as in the [quickstart](/develop/docs/dev-catalog-quickstart#before-you-start), and use `CATALOG_PATH` as the subtree to compare.

## List the Catalog snapshots

`tx.catalog_snapshots()` returns the snapshots of the Catalog's database, `vast-big-catalog-bucket`, as `Bucket` objects named `vast-big-catalog-bucket/.snapshot/<name>`. The copies of the Catalog table are the ones named `bc_table_` followed by the date and time; the list may include other snapshots too. Sort them by name to put them in time order:

```
import vastdb

session = vastdb.connect()
with session.transaction() as tx:
    names = sorted(
        s.name.split("/.snapshot/")[-1]
        for s in tx.catalog_snapshots()
        if s.name.split("/.snapshot/")[-1].startswith("bc_table_")
    )

print(len(names), "Catalog snapshots")
for name in names:
    print(name)
```

```
2 Catalog snapshots
bc_table_2026-10-09_16_51_02
bc_table_2026-10-09_17_06_02
```

How many there are depends on the Catalog's schedule: one per interval, each kept for the "keep catalog copies" period. With the default schedule (every 30 minutes, kept for 30 minutes) you'll see only one or two. To compare across hours or days, ask your platform team for a longer retention; see [snapshots and freshness](/develop/docs/dev-catalog-concepts-index#snapshots-and-freshness).

## Read a snapshot

Pass a snapshot to `tx.catalog()` to query the Catalog as it was then. Everything else works as on the current Catalog: columns, filters and `search_path`. Snapshots are read-only.

```
import os

import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

session = vastdb.connect()
with session.transaction() as tx:
    snapshots = sorted(
        (s for s in tx.catalog_snapshots() if s.name.split("/.snapshot/")[-1].startswith("bc_table_")),
        key=lambda s: s.name,
    )
    oldest = snapshots[0]
    rows = tx.catalog(snapshot=oldest).select(
        columns=["size"],
        predicate=(_.search_path == path) & (_.element_type == "FILE"),
    ).read_all()

print(oldest.name.split("/")[-1], rows.num_rows, "files,", sum(rows["size"].to_pylist()), "bytes")
```

```
bc_table_2026-10-09_16_51_02 23 files, 57794560 bytes
```

## Find deleted and added files

To find what changed, read the same columns with the same filter from two snapshots, and compare the full paths. A path in the older snapshot but not the newer one was deleted (or moved away); a path only in the newer one was added (or moved in).

![A timeline with two Catalog snapshots, older and newer, each a copy of the Catalog table kept for its retention period. Read the same columns with the same search_path from both, and compare full paths (parent_path plus name). A path only in the older copy was deleted or moved away: scratch/to-delete.tmp. A path only in the newer copy was added or moved in: projects/gemini/sales-5.csv. A renamed file keeps its phandle, so matching on phandle pairs the old and new paths: archive/2025/notes.txt became notes-2025.txt. Changes made and undone between two snapshots appear in neither.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/cat-change-detection.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A56%3A06Z&se=2026-11-08T09%3A56%3A06Z&sr=c&sp=r&sig=84lmfc%2BhvQeRhAdNYMm0yzbOJctSX%2B0IjBhdWEADgPY%3D)

*Two Catalog snapshots compared: the example output below, as a picture.*

This example needs at least two Catalog snapshots, with a file deleted and a file added under your path between them:

```
import os

import pyarrow.compute as pc
import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

def full_paths(table):
    return pc.binary_join_element_wise(table["parent_path"], table["name"], "")

session = vastdb.connect()
with session.transaction() as tx:
    snapshots = sorted(
        (s for s in tx.catalog_snapshots() if s.name.split("/.snapshot/")[-1].startswith("bc_table_")),
        key=lambda s: s.name,
    )
    older, newer = snapshots[0], snapshots[-1]
    query = dict(
        columns=["parent_path", "name"],
        predicate=(_.search_path == path) & (_.element_type == "FILE"),
    )
    before = full_paths(tx.catalog(snapshot=older).select(**query).read_all())
    after = full_paths(tx.catalog(snapshot=newer).select(**query).read_all())

deleted = pc.filter(before, pc.invert(pc.is_in(before, value_set=after)))
added = pc.filter(after, pc.invert(pc.is_in(after, value_set=before)))

print("from", older.name.split("/")[-1], "to", newer.name.split("/")[-1])
print("deleted:", sorted(deleted.to_pylist())[:10])
print("added:", sorted(added.to_pylist())[:10])
```

```
from bc_table_2026-10-09_16_51_02 to bc_table_2026-10-09_17_06_02
deleted: ['/cat-data/archive/2025/notes.txt', '/cat-data/scratch/to-delete.tmp']
added: ['/cat-data/archive/2025/notes-2025.txt', '/cat-data/projects/gemini/sales-5.csv']
```

A few things to keep in mind:

- **Compare like with like.** Use the same `search_path` and the same filters for both snapshots. A filter on a value that changes, such as `mtime`, makes unchanged files look added or deleted.
- **Changes between snapshots only.** A file created and deleted between two snapshots appears in neither.
- **Sizes and times too.** To find modified files, add `mtime` or `size` to the columns and compare them for paths that are in both.

## Find moved files

A rename changes an element's path, so the comparison above shows a moved file as one deletion and one addition. The `phandle` column identifies the element itself, and a renamed file keeps it, so matching on it pairs each old path with its new one:

```
import os

import pyarrow.compute as pc
import vastdb
from ibis import _

path = os.environ.get("CATALOG_PATH", "/")

def paths_by_handle(table):
    flat = table.flatten()
    return flat.select(["phandle.clone_id", "phandle.handle_id"]).append_column(
        "path", pc.binary_join_element_wise(flat["parent_path"], flat["name"], "")
    )

session = vastdb.connect()
with session.transaction() as tx:
    snapshots = sorted(
        (s for s in tx.catalog_snapshots() if s.name.split("/.snapshot/")[-1].startswith("bc_table_")),
        key=lambda s: s.name,
    )
    query = dict(
        columns=["phandle", "parent_path", "name"],
        predicate=(_.search_path == path) & (_.element_type == "FILE"),
    )
    before = paths_by_handle(tx.catalog(snapshot=snapshots[0]).select(**query).read_all())
    after = paths_by_handle(tx.catalog(snapshot=snapshots[-1]).select(**query).read_all())

keys = ["phandle.clone_id", "phandle.handle_id"]
joined = before.join(after, keys=keys, left_suffix="_before", right_suffix="_after")
moved = joined.filter(pc.not_equal(joined["path_before"], joined["path_after"]))
for row in moved.slice(0, 10).to_pylist():
    print(row["path_before"], "->", row["path_after"])
```

```
/cat-data/archive/2025/notes.txt -> /cat-data/archive/2025/notes-2025.txt
```

Moves into or out of your `search_path` show up only on one side, so run the comparison on a path that contains both the old and the new location.

## Keep your own history

Catalog snapshots are deleted when their retention ends. If you need a longer history, for example a daily capacity trend, run your report on a schedule and insert the result into a table of your own in VAST DataBase. See [ingest](/develop/docs/dev-database-guides-ingest).

## See also

- [Concepts: snapshots and freshness](/develop/docs/dev-catalog-concepts-index#snapshots-and-freshness).
- [Snapshots of your own tables](/develop/docs/dev-database-operations-snapshots-and-replication).
- [Capacity reports](/develop/docs/dev-catalog-guides-capacity-reports).