---
title: "Integrations"
slug: "dev-dataengine-integrations-index"
description: "Learn to connect DataEngine functions and pipelines to VAST DataBase, observability stacks, and more, using SDKs, REST API, and Kafka protocol."
tags: ["DataEngine"]
updated: 2026-10-30T18:19:01Z
published: 2026-10-30T18:19:01Z
canonical: "docs.vastdata.com/dev-dataengine-integrations-index"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Integrations

*Applies to VAST Cluster 5.5.*

These pages show how to connect DataEngine functions and pipelines to other systems: VAST DataBase tables, the dead-letter topic, your own observability stack, workflow orchestrators and lineage tools.

**None of these is a built-in DataEngine integration in VAST Cluster 5.5.** DataEngine has no connectors for them. Each page is a pattern: code you write in your handler, or a client you run next to DataEngine, using the SDKs, the REST API and the Kafka protocol.

| Page | What it shows |
| --- | --- |
| [VAST DataBase and Vector Search](/develop/docs/dev-dataengine-integrations-vast-database-and-vector-search) | Writing rows to a VAST DataBase table from a handler with the `vastdb` SDK: dependencies, S3 keys as a secret, connecting in `init`, and reading the table back. Embeddings follow the same pattern. |
| [Dead-letter topic](/develop/docs/dev-dataengine-integrations-dead-letter-topic) | A Kafka consumer that reads the tenant's dead-letter topic, decodes why each event failed, and alerts on it or replays it. |
| [OpenTelemetry](/develop/docs/dev-dataengine-integrations-opentelemetry) | Sending your function's telemetry to an OpenTelemetry backend you run, as well as to DataEngine. |
| [Orchestrators](/develop/docs/dev-dataengine-integrations-orchestrators) | Driving and watching pipelines from Airflow and similar orchestrators, through the REST API. |
| [OpenLineage and OpenMetadata](/develop/docs/dev-dataengine-integrations-openlineage-and-openmetadata) | Emitting OpenLineage run events from a handler, from the object that triggered it to the table or object it wrote. |

## Before you start

Every pattern runs inside your function or next to it, so it needs what a function needs to reach the other system:

- **Network access from the compute cluster.** A function's pods connect to databases, brokers and backends from the compute cluster, not from your machine. Ask your platform team which addresses the compute cluster can reach.
- **Credentials as secrets.** Pass keys and tokens as DataEngine secrets, and settings as environment variables without the reserved `VAST_` prefix. See [security](/develop/docs/dev-dataengine-operations-security#secrets) and [write a function](/develop/docs/dev-dataengine-guides-write-a-function#environment-variables-and-secrets).
- **Dependencies in the image.** Clients such as `vastdb` or `openlineage-python` go in `requirements.txt`, under the builder's constraints for the runtime's own packages. See [write a function](/develop/docs/dev-dataengine-guides-write-a-function#dependencies).

## See also

- [CI/CD and automation](/develop/docs/dev-dataengine-operations-ci-cd): building and deploying functions and pipelines from CI, with `vastde` and the REST API.
- [Observability](/develop/docs/dev-dataengine-guides-observability): the logs, traces and metrics DataEngine collects itself.
- VAST DataBase [Python SDK](/develop/docs/dev-database-integrations-python-sdk), VAST Vector Search [Python SDK](/develop/docs/dev-vector-search-integrations-python-sdk) and VAST Event Broker [integrations](/develop/docs/dev-event-broker-integrations-index).