--- title: "Setup" slug: "dev-event-broker-operations-setup" description: "Set up your Kafka view with VIP pools and security settings. Learn how to request configurations and verify client connections effectively." updated: 2026-10-30T18:19:06Z published: 2026-10-30T18:19:06Z canonical: "docs.vastdata.com/dev-event-broker-operations-setup" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Setup *Applies to VAST Cluster 5.5.* Your platform team creates the broker: a Kafka view, its VIP pool, and its security settings. This page lists what to ask them for, gives you a request you can copy, and shows how to check that your Kafka client and the VAST DataBase SDK can both reach it. ## What you need | You need | What it is | Who provides it | | --- | --- | --- | | A Kafka view | A view with the Kafka protocol enabled. It's your broker. The view also has the S3 and DATABASE protocols: DATABASE is what lets you read topics as tables. Its S3 bucket name is also the name of the database that holds your topics. | Platform team | | Bootstrap addresses | Three virtual IPs from the view's VIP pool, as `<vip>:<port>`. | Platform team | | Network access to the whole pool | Your clients connect to every virtual IP in the pool, not only the bootstrap ones. Open the Kafka ports to all of them. | Platform team, network team | | The listener and port | **9092** for unencrypted connections, **9093** for TLS. A view can allow either or both. | Platform team | | An authentication method | None, SASL/PLAIN, or mTLS (5.5). See [security](/develop/docs/dev-event-broker-operations-security). | Platform team | | Credentials | For SASL/PLAIN, a VAST **local** user and its password (not an Active Directory or LDAP user, and not S3 keys). For mTLS, a client certificate and key. | Platform team | | A CA certificate | For TLS: the certificate of the CA that signed the cluster's Kafka certificate. | Platform team | | Permissions | If the view requires authorization, an identity policy with the Kafka actions you need. See [security](/develop/docs/dev-event-broker-operations-security#permissions). | Platform team | | Topics, or the right to create them | Topics aren't created automatically. Ask for the topics, or for permission to create them with the Kafka Admin API. See [topics](/develop/docs/dev-event-broker-guides-topics). | Platform team, or you | | A Kafka client | A supported client, such as the Confluent Kafka Python client 2.4 to 2.8 these docs use: `pip install "confluent-kafka>=2.4,<2.9"`. See [supported clients](/develop/docs/dev-event-broker-reference-kafka-compatibility#supported-clients). The aiokafka client isn't supported. | You | | For reading topics as tables | An S3 access key pair, a VAST DataBase endpoint, and read access to the Kafka view's bucket. See [setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites). | Platform team | | For blob expansion | The `s3:TabularCreateBlobExpansion`, `s3:TabularAlterBlobExpansion`, `s3:TabularDropBlobExpansion` and `s3:TabularGetBlobExpansion` actions, and a schema you can create tables in. See [blob expansion](/develop/docs/dev-event-broker-guides-blob-expansion). | Platform team | **Admin task: send to your platform team.** The setup steps are in the KB: - [Configuring VAST Event Broker](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-vast-event-broker): the VIP pool, the bucket owner and the Kafka view. - [Configuring user authentication](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-user-authentication-for-the-vast-event-broker-view): TLS and SASL/PLAIN, and the cluster's Kafka TLS certificate. - [Configuring mTLS authentication](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-mtls-authentication-for-the-vast-event-broker-view) (5.5). - [Configuring user authorization](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-user-authorization-for-the-vast-event-broker-view). - [Granting Kafka permissions in an identity policy](/docs-5-5/docs/publishing-events-to-vast-event-broker#granting-kafka-permissions-in-an-identity-policy). - Tenant administrators: the [tenant guide version](/docs-5-5/docs/publishing-events-to-vast-event-broker). The VIP pool needs a cluster administrator. ![Six steps, in order, then your application connects. 1, platform team: a VIP pool with the Protocols role and at least one virtual IP per CNode, created by a cluster administrator; a Kafka view has exactly one, which can be modified but not replaced. 2, platform team: the Kafka view on a new path, with its bucket owner user and the VIP pool; enabling Kafka also enables S3 and DATABASE, and the bucket name is the name of the database that holds your topics. 3, platform team: connections and authentication: unencrypted on port 9092, TLS on port 9093, or both; no authentication, SASL/PLAIN with a VAST local user, or mTLS; and the CA certificate for TLS. 4, platform team: if authorization is on, an identity policy with the Kafka actions you need, such as KafkaRead and KafkaWrite on topics and KafkaRead on consumer groups. 5, platform team or you: topics, with partition counts, which are fixed when a topic is created, and retention, or the right to create topics with the Kafka Admin API. 6, platform team: for reading topics as tables and for blob expansion, an S3 access key pair, the database endpoint, read access to the view's bucket and the s3:Tabular*BlobExpansion actions. Then your application connects: bootstrap.servers is three virtual IPs from the pool, as vip:port, every virtual IP in the pool must be reachable, and producers set compression.type to none and enable.idempotence to false.](https://cdn.us.document360.io/89b91113-5dc6-49db-86ec-57b16dc8ce56/Images/Documentation/eb-setup-flow.svg?sv=2026-02-06&spr=https&st=2026-11-08T08%3A55%3A19Z&se=2026-11-08T09%3A55%3A19Z&sr=c&sp=r&sig=%2BwqseRb5jpObEYbxkDUGtLRllIDv0wek1sMhnzP12AQ%3D) *What your platform team sets up for a broker, in order, before your Kafka client connects.* ## Decide before the view is created Some choices can't be changed later, so agree on them first: - **The VIP pool.** A Kafka view has exactly one VIP pool, with at least one virtual IP per CNode. Once the view exists, the pool can be modified but not replaced by another one. - **Kafka on a new view.** The Kafka protocol can't be added to an existing view, so the broker is always a new view. It can't be removed from a view that holds topics either. - **Partition counts.** Each topic's partition count is fixed when the topic is created, up to 20,000 per topic and 200,000 per view. Size it for the most consumers you'll run in one group. - **Security.** Whether clients connect with TLS, and how they authenticate. If you want the topics protected per user, ask for authorization to be turned on. Without it, any client that can connect can read, write and delete every topic. If nothing is set, a new Kafka view accepts unencrypted connections with no authentication, and doesn't check permissions. ## What to send your platform team Copy this, and fill in the parts in angle brackets: ``` Please set up a VAST Event Broker (Kafka) view for : - Kafka view on a new path, tenant , with its own VIP pool (role Protocols, at least one VIP per CNode). - Connections: . - Authentication: . - SASL/PLAIN: a VAST local user and its password. - mTLS: a client certificate for user , and the CA certificate. - Authorization: . If on, an identity policy for that allows KafkaRead and KafkaWrite on topics , KafkaRead on consumer groups , and . - Topics: , or permission to create them. - For SQL/SDK reads of the topics: an S3 access key pair, the database endpoint, and read access to the view's bucket. - For blob expansion: s3:Tabular*BlobExpansion actions on the bucket, and a schema we can create tables in. Please send back: the bucket name, three bootstrap VIPs, the port(s), the CA certificate (for TLS), and the credentials. ``` ## Client configuration Every client needs the bootstrap addresses. Producers also need two settings for VAST Event Broker, whatever their language: no compression, and no idempotence. ``` import os conf = { "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"], # e.g. ":9092,:9092,:9092" } producer_conf = { **conf, "compression.type": "none", # compressed batches are rejected "enable.idempotence": False, # idempotent producing isn't supported } ``` Set both explicitly in every producer, even where they're your client's default. The settings for TLS, SASL/PLAIN and mTLS are on the [security](/develop/docs/dev-event-broker-operations-security#client-settings) page. ## Check that it works These checks read and create nothing. Set the environment variables first: ``` export KAFKA_BOOTSTRAP=":9092,:9092,:9092" # Only if the view uses SASL/PLAIN: export KAFKA_USERNAME="" export KAFKA_PASSWORD="" ``` ### The Kafka connection This asks the broker for its metadata. It works on the unencrypted port, with or without SASL/PLAIN. For TLS, add the settings from [security](/develop/docs/dev-event-broker-operations-security#client-settings). ``` import os from confluent_kafka.admin import AdminClient conf = {"bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"]} if os.environ.get("KAFKA_USERNAME"): conf.update({ "security.protocol": "SASL_PLAINTEXT", "sasl.mechanism": "PLAIN", "sasl.username": os.environ["KAFKA_USERNAME"], "sasl.password": os.environ["KAFKA_PASSWORD"], }) admin = AdminClient(conf) metadata = admin.list_topics(timeout=10) brokers = list(metadata.brokers.values()) bootstrap = os.environ["KAFKA_BOOTSTRAP"].split(",") print("brokers:", len(brokers), "on port(s)", sorted({b.port for b in brokers})) print("every bootstrap address is a broker:", all(any(f"{b.host}:{b.port}" == addr for b in brokers) for addr in bootstrap)) print("topics:", len(metadata.topics)) ``` ``` brokers: 3 on port(s) [9092] every bootstrap address is a broker: True topics: 0 ``` You should see one broker per virtual IP in the pool. If the call times out, check the address and port, and that your network reaches the virtual IPs. If the connection closes straight away, check the authentication settings; see [troubleshooting](/develop/docs/dev-event-broker-troubleshooting-index). ### The topic tables This lists the topics as tables, through the VAST DataBase SDK. It needs the S3 key pair and database endpoint, and the bucket name of the Kafka view: ``` export AWS_ACCESS_KEY_ID="" export AWS_SECRET_ACCESS_KEY="" export AWS_S3_ENDPOINT_URL="http://" export EVENTS_BUCKET="" ``` ``` import os import vastdb session = vastdb.connect() with session.transaction() as tx: schema = tx.bucket(os.environ["EVENTS_BUCKET"]).schema("kafka_topics") print("topic tables:", sorted(schema.tablenames())) ``` ``` topic tables: [] ``` The list matches the topics from the Kafka check, if your Kafka user can see all of them. ## See also - [Security](/develop/docs/dev-event-broker-operations-security): authentication modes, client settings and permissions. - [Quickstart](/develop/docs/dev-event-broker-quickstart): produce, consume and query a topic. - [Topics](/develop/docs/dev-event-broker-guides-topics): create topics with the Kafka Admin API. - [VAST DataBase setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites): keys, endpoint and Tabular permissions for the SDK.