Transactions

Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.

This page explains how VAST DataBase transactions behave: what a transaction sees, when its changes become visible, and what happens when two transactions write the same data. For code patterns, such as retries and handling conflicts, see transactions in practice.

What a transaction sees

When a transaction begins, it takes a snapshot. Every read in the transaction sees:

everything committed before the transaction began, and
the transaction's own changes.

It doesn't see changes from transactions that are still open or that rolled back, or from transactions that commit after it began. Readers don't wait for writers, and writers don't wait for readers. This is snapshot isolation. To see newer commits, start a new transaction.

Three timelines. Transaction R begins and takes a snapshot. Rows committed before it began are visible to it, and so are its own changes. Transaction W begins later, inserts a row and commits while R is still open: R doesn't see that row, and nobody waits for anybody. Transaction N begins after W's commit and sees W's row. To see newer commits, start a new transaction.

A transaction sees its snapshot and its own changes; later commits need a new transaction.

Reads through a bucket snapshot see the data as of that snapshot. They don't see your transaction's own uncommitted changes.

When changes become visible

A transaction's changes become visible to others only when it commits, and all at once. Once a commit has started, it completes even if a node fails partway through. Transactions that begin after the commit see all of its changes.

DDL (creating, renaming and dropping schemas, tables and columns) is part of the same transaction as your row changes. They commit or roll back together, and other transactions don't see uncommitted DDL. See schema management.

One exception: delete_partitions() isn't part of your transaction. It runs on its own and takes effect at once, and a rollback doesn't undo it (database partitioning says it's "non-ACIDic").

When two transactions conflict

The cluster checks for conflicts when you write, not when you commit. Two writes conflict only when they change the same cells: the same column of the same rows.

Two open transactions...	Conflict?
Insert rows into the same table	No. New rows never conflict.
Update different rows	No
Update different columns of the same row	No. Both changes are kept.
Update the same column of the same row	Yes
Delete the same row	Yes
One deletes a row, the other updates it	No. Both commit, and the row ends up deleted.
Create, rename or drop the same schema, table or column name	Yes

Who wins

Whoever writes first, the transaction that began first wins.

Row changes: the older transaction wins. "Older" means the transaction that began first, not the one that wrote first.

If your transaction is the younger one and writes second, your call fails at once with Conflict (409, code TabularWwc).
If your transaction is the younger one but wrote first, the cluster rolls it back when the older one writes. You find out at your next request or at commit, which fails with BadRequest (400, code TabularTxAlreadyRollbacked).
If another transaction committed a change to the same cells after your transaction began, your write fails with TabularWwc.

In every case, your transaction has already been rolled back. Retry the whole transaction.

Names in DDL: the first transaction to claim a name wins, whatever its age. A second transaction that creates, renames or drops the same schema, table or column name gets TabularWwc straight away. Adding different columns to the same table from two transactions works.

The cluster's background work, such as sorting, gives way to your transactions.

Dropping a table that's being written to

Dropping a table doesn't wait for, or conflict with, other transactions that are writing to it. When the drop commits, the table is gone, along with any rows those transactions inserted. Their commits still report success. Their later requests to the table fail with InternalServerError (500), message Permission check failed for .... Coordinate drops with whatever writes to the table.

Transactions that are left open

The cluster rolls back transactions that stay idle for too long, and transactions that stay open for too long, even if they're active. A later request in that transaction then fails with TabularTxAlreadyRollbacked. Keep transactions short, and don't pause inside one, for example while waiting for user input.

Permissions for transactions

Beginning, committing and rolling back a transaction are refused only if a policy explicitly denies them. That's why the transaction actions (s3:TabularBeginTransaction and the others) aren't strictly needed. Granting them anyway is recommended, and harmless. See setup prerequisites.

What this means for your code

Retry whole transactions, not individual calls. A rolled-back transaction leaves nothing behind, except delete_partitions() and anything outside the database.
Expect conflicts only on shared cells. Parallel loaders that insert into the same table don't conflict with each other.
Don't rely on serializability. The cluster checks only whether two writes overlap. Two transactions that each read data and then write different cells can both commit, even if each one's decision depended on what the other changed.
A delete beats a concurrent update of the same row.
Coordinate DDL. Creating the same table from two jobs at once fails for one of them; use fail_if_exists=False and handle Conflict.
Keep transactions short, so the cluster doesn't roll them back.