--- title: "Python SDK" slug: "dev-database-integrations-python-sdk" description: "Explore the vastdb Python SDK for VAST DataBase, enabling seamless data handling with PyArrow schemas, transactions, and efficient error management." updated: 2026-10-30T18:18:46Z published: 2026-10-30T18:18:46Z canonical: "docs.vastdata.com/dev-database-integrations-python-sdk" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Python SDK *Applies to VAST Cluster 5.5. Examples use the VAST DataBase Python SDK 2.1.0.* The `vastdb` Python SDK is the native client for VAST DataBase. It talks to the cluster directly, and it uses [Apache Arrow](https://arrow.apache.org/docs/python/) for everything: table schemas are PyArrow schemas, inserts take Arrow tables, and queries return Arrow record batches. This page explains how the SDK's objects fit together and the idioms that keep your code short and correct. For every class, parameter and default, see the [Python SDK reference](/develop/docs/dev-database-reference-sdk). Run the code blocks in order in one Python session. Below each block is the output you should see. ## Install ``` pip install "vastdb==2.1.0" ``` The SDK needs Python 3.10 or later. For which SDK versions work with which cluster versions, see [versions](/develop/docs/dev-database-reference-versions). Before you can connect, your platform team has to give you an endpoint, keys and permissions; see [setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites). ## The object lifecycle | Object | How you get it | Lifetime | | --- | --- | --- | | `Session` | `vastdb.connect()` | Create one per process and reuse it. Creating it contacts the cluster to check its version. | | `Transaction` | `session.transaction()`, used as a `with` block | One unit of work. It commits when the block ends normally, and rolls back if an exception leaves the block. | | `Bucket`, `Schema`, `Table` | `tx.bucket(name)`, `.schema(name)`, `.table(name)` | Bound to the transaction they came from. Don't keep them after the block ends. | | `RecordBatchReader` from `select()` | `table.select(...)` | Also bound to the transaction. Read it before the block ends. | | `TableMetadata` | `TableMetadata(TableRef(...))`, then `.load(tx)` | Not bound to a transaction. Keep it, and reuse it in later transactions. | A database in VAST is an S3 bucket on a Database-enabled view, so the SDK calls it a bucket. For the concepts behind these objects, see [concepts](/develop/docs/dev-database-concepts-index). Connect, using the environment variables from the [quickstart](/develop/docs/dev-database-quickstart#2-connect): ``` import os import pyarrow as pa import vastdb from vastdb import errors session = vastdb.connect() BUCKET = os.environ["VASTDB_BUCKET"] SCHEMA = "integ_python_sdk" print("Cluster version:", ".".join(str(n) for n in session.features.vast_version)) ``` ``` Cluster version: 5.5.0.1 ``` `vastdb.connect()` takes the same arguments as `Session`: you can pass `endpoint`, `access` and `secret` explicitly instead of using the environment. It also takes `timeout` (per HTTP request), `ssl_verify` and `backoff_config` (see [Retries](/develop/docs/dev-database-integrations-python-sdk#retries-and-errors)). ## Create-or-get, and get-or-none Most setup code wants "create it unless it exists". Pass `fail_if_exists=False` to `create_schema()` and `create_table()` to get the existing object back instead of an error. Code that reads wants the opposite: pass `fail_if_missing=False` to `schema()` and `table()` to get `None` instead of an exception. ``` columns = pa.schema([("id", pa.int64()), ("name", pa.utf8())]) with session.transaction() as tx: bucket = tx.bucket(BUCKET) schema = bucket.create_schema(SCHEMA, fail_if_exists=False) table = schema.create_table("people", columns, fail_if_exists=False) print("table:", table.name) print("missing table:", schema.table("no_such_table", fail_if_missing=False)) print("tables:", schema.tablenames()) ``` ``` table: people missing table: None tables: ['people'] ``` With `fail_if_exists=False`, an existing table is returned as it is. Its columns aren't compared with the ones you passed, so check `table.arrow_schema` if that matters to you. To list tables, prefer `schema.tablenames()` to `schema.tables()`. `tables()` opens every table, and that loads each one's columns and statistics. ## Commit and roll back A transaction commits only if its block ends without an exception. If anything raises inside the block, everything the transaction did is rolled back: ``` class Abort(Exception): pass try: with session.transaction() as tx: tx.bucket(BUCKET).schema(SCHEMA).table("people").insert( pa.table({"id": [1], "name": ["Ada"]}, schema=columns)) raise Abort("changed my mind") except Abort: pass with session.transaction() as tx: print("rows after rollback:", tx.bucket(BUCKET).schema(SCHEMA).table("people").select().read_all().num_rows) ``` ``` rows after rollback: 0 ``` When a transaction rolls back, the SDK logs `rolling back txid=... due to:` and the traceback at `ERROR` level, through the `vastdb.transaction` logger. If your program hasn't configured logging, Python prints that message to stderr, even when you catch the exception. Configure logging in your application to control where it goes. Catch exceptions **outside** the `with` block when you want the transaction's work undone. If you catch them inside, the block ends normally and the transaction commits whatever was already written. For retry and concurrency patterns, see [transactions in practice](/develop/docs/dev-database-guides-transactions-in-practice). ## Reuse table metadata `schema.table(name)` makes several requests each time: it checks the table exists, then loads its columns and statistics. In a loop that opens the same table in many transactions, load a `TableMetadata` once and rebuild the table handle from it: ``` from vastdb import TableRef from vastdb.table_metadata import TableMetadata md = TableMetadata(TableRef(BUCKET, SCHEMA, "people")) with session.transaction() as tx: md.load(tx) for start in range(0, 30, 10): with session.transaction() as tx: people = tx.table_from_metadata(md) people.insert(pa.table({ "id": list(range(start, start + 10)), "name": [f"user-{i}" for i in range(start, start + 10)], }, schema=columns)) with session.transaction() as tx: people = tx.table_from_metadata(md) print(people.select(columns=["id"]).read_all().num_rows, "rows") ``` ``` 30 rows ``` `load(tx)` loads the columns, the table type and the statistics. `select()` needs the table type, so a `TableMetadata` you only built with `arrow_schema=` is enough for `insert()` but not for `select()`. `tx.table_from_metadata()` returns a table without the DDL methods (`drop()`, `add_column()` and so on). After you change the table's columns, load the metadata again. See [TableMetadata and TableRef](/develop/docs/dev-database-reference-sdk#tablemetadata-and-tableref). ## Configuration objects The SDK groups settings into small dataclasses in `vastdb.config`. You pass them where they're used: | Class | Pass it to | Use it for | | --- | --- | --- | | `QueryConfig` | `select(config=...)`, `select_splits(config=...)` | Splits, endpoints and threads for a query. See [querying](/develop/docs/dev-database-guides-querying#tune-a-query-with-queryconfig). | | `ImportConfig` | `import_files(config=...)` | Import concurrency, endpoints, and key columns. See [ingest](/develop/docs/dev-database-guides-ingest#import-settings). | | `BufferedInsertConfig` | `session.buffered_inserter(..., flush_config=...)` | When to flush, and whether to use one transaction. See [ingest](/develop/docs/dev-database-guides-ingest#load-a-partitioned-table). | | `BackoffConfig` | `vastdb.connect(backoff_config=...)` | How the SDK retries requests. See [below](/develop/docs/dev-database-integrations-python-sdk#retries-and-errors). | | `SessionConfig` | `vastdb.connect(config=...)` | Session-wide default data endpoints for queries and imports. | `QueryConfig`, `ImportConfig`, `BufferedInsertConfig` and `BackoffConfig` are also importable straight from `vastdb`. `SessionConfig` is only in `vastdb.config`. Invalid values, such as an empty `data_endpoints` list, raise `ValueError` when you create the object. ## Retries and errors The SDK retries a request by itself only when the cluster asks it to slow down (`errors.Slowdown`), or after a connection failure on a request that's safe to repeat, such as a read. `BackoffConfig` controls those retries; by default the SDK makes up to 10 attempts within 60 seconds. Everything else is raised to your code. All SDK exceptions are in `vastdb.errors`. Catch the family you can handle: | Catch | When | | --- | --- | | `errors.Missing` (`MissingBucket`, `MissingSchema`, `MissingTable`, ...) | An object doesn't exist. These carry the names as attributes. | | `errors.Exists` (`SchemaExists`, `TableExists`) | A create call found the object already there. | | `errors.HttpError` (`BadRequest`, `Forbidden`, `NotFound`, `Conflict`, ...) | The cluster rejected a request. Each one has `code`, `message`, `status`, `method`, `url` and `headers`. | | `errors.InvalidArgument` (`TooLargeRequest`, `TooWideRow`) | The SDK refused the input. | | `errors.NotSupported` (`NotSupportedVersion`, `NotSupportedSchema`, ...) | The cluster version or a column type doesn't support the operation. | | `NotImplementedError` | A predicate the SDK can't send. See [querying](/develop/docs/dev-database-guides-querying#rewrite-filters-that-dont-push-down). | ``` from vastdb.config import QueryConfig with session.transaction() as tx: schema = tx.bucket(BUCKET).schema(SCHEMA) try: schema.table("no_such_table") except errors.MissingTable as e: print(type(e).__name__, "->", e.schema, e.table) people = schema.table("people") try: people.select(config=QueryConfig(queue_priority=-1)).read_all() except errors.HttpError as e: print(type(e).__name__, "->", e.status, e.code) ``` ``` MissingTable -> integ_python_sdk no_such_table BadRequest -> 400 InvalidArgument ``` Notice that the second error was caught *inside* the transaction, and the transaction then committed. That's fine here because the failed query didn't change anything. The [error reference](/develop/docs/dev-database-troubleshooting-errors) lists every message with its cause and fix. Never print `e.url` or `e.headers` into shared logs without checking them: they contain your endpoint's address. The SDK also logs every failed request at `WARNING` level, as `RPC failed: {...}` with the same fields, through the `vastdb.errors` logger. To see what the SDK is doing, turn on its logger: ``` import logging logging.basicConfig(level=logging.INFO) logging.getLogger("vastdb").setLevel(logging.DEBUG) ``` ## Check what the cluster supports `session.features` describes the connected cluster. The SDK checks it before it uses a version-dependent feature, and raises `errors.NotSupportedVersion` without sending anything if the cluster is too old. You can check first yourself: ``` try: session.features.check_partitioned() print("partitioned tables: supported") except errors.NotSupportedVersion: print("partitioned tables: not supported by this cluster") ``` ``` partitioned tables: supported ``` [Version-gated features](/develop/docs/dev-database-reference-sdk#version-gated-features) lists the checks and the cluster version each one needs. ## Threads and processes The SDK uses one session from several threads itself: a query runs one worker thread per data endpoint, and the buffered inserter flushes from your thread pool, opening transactions on the same session. ## Tests against a real cluster The SDK's own test suite shows a pattern that works well: give each test run its own schema, create it with `fail_if_exists=False`, and drop every table in it, then the schema, when the run ends. ``` import pytest import vastdb @pytest.fixture def schema_name(): session = vastdb.connect() name = "test_myapp" yield name with session.transaction() as tx: schema = tx.bucket(BUCKET).schema(name, fail_if_missing=False) if schema is not None: for table_name in schema.tablenames(): schema.table(table_name).drop() schema.drop() ``` ## Clean up ``` with session.transaction() as tx: schema = tx.bucket(BUCKET).schema(SCHEMA) schema.table("people").drop() schema.drop() print("Dropped", SCHEMA) ``` ``` Dropped integ_python_sdk ``` ## Next steps - [Ingest](/develop/docs/dev-database-guides-ingest) and [querying](/develop/docs/dev-database-guides-querying): the SDK's data paths in depth. - [Arrow, pandas and Polars](/develop/docs/dev-database-integrations-arrow-pandas-polars): moving data between the SDK and DataFrame libraries. - [Python SDK reference](/develop/docs/dev-database-reference-sdk): the full API.