--- title: "Security" slug: "dev-event-broker-operations-security" description: "Learn about security in VAST Event Broker, including Kafka views, authentication methods, and client settings for optimal connection management." updated: 2026-10-30T18:19:06Z published: 2026-10-30T18:19:06Z canonical: "docs.vastdata.com/dev-event-broker-operations-security" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Security *Applies to VAST Cluster 5.5.* Security in VAST Event Broker is set per Kafka view by your platform team: which connections the view accepts (unencrypted, TLS or both), how clients authenticate on each (none, SASL/PLAIN or mTLS), and whether identity policies decide what each user may do. This page explains what each choice means for your clients, gives the client settings for each mode, and lists the permissions to ask for. ## Connections and authentication A Kafka view can accept two kinds of connection, each on its own port, and sets an authentication method for each: | Connection | Port | Authentication methods | | --- | --- | --- | | Unencrypted | 9092 | None, or SASL/PLAIN | | TLS | 9093 | None, SASL/PLAIN, or mTLS (5.5) | A view can allow one kind or both. Each kind has exactly one method, so on TLS a client uses SASL/PLAIN or mTLS, not both. If a client uses a kind the view doesn't allow, the broker accepts the TCP connection and closes it at the first request, before it answers even the version request. If your platform team sets nothing, a new Kafka view accepts unencrypted connections without authentication. ### TLS TLS uses one Kafka certificate for the whole cluster. The cluster gives clients IP addresses, not host names, so the certificate lists every virtual IP of every Kafka view's VIP pool as an IP address in its Subject Alternative Names. It's issued by an internal CA, because public CAs don't issue certificates for private IP addresses. Your client needs that CA's certificate: without it, the TLS handshake fails. Hostname verification, which `confluent-kafka` does by default, passes against the IP-address entries. The broker negotiates TLS 1.2. A client that allows only TLS 1.3 fails the handshake. To look at the certificate the broker presents: ``` openssl s_client -connect :9093 -showcerts ``` If your platform team adds another Kafka view later, they reissue the certificate with its virtual IPs. Clients keep working as long as they trust the same CA. ### SASL/PLAIN SASL/PLAIN works on both unencrypted and TLS connections. The user name and password are those of a **VAST local user** in the view's tenant. Active Directory and LDAP users aren't supported. S3 access keys don't work here. Use SASL/PLAIN over TLS: on an unencrypted connection, the password crosses the network in clear text. Things to know: - `PLAIN` is the only SASL mechanism. The broker answers a request for SCRAM, GSSAPI (Kerberos) or OAUTHBEARER by listing `PLAIN` as its only mechanism, then closes the connection. `confluent-kafka` logs `SASL authentication error: SaslAuthenticateRequest failed: Local: Broker transport failure`. - When the view requires SASL/PLAIN, the broker answers only the version and SASL handshake requests before authentication. Any other request closes the connection, with no error code. `confluent-kafka` logs `Disconnected: verify that security.protocol is correctly configured, broker might require SASL authentication`. - When the view doesn't use SASL on that connection kind, a SASL handshake gets `ILLEGAL_SASL_STATE`. `confluent-kafka` reports it as a broker transport failure. - A wrong password, or a user name that isn't a local user, gets `SASL_AUTHENTICATION_FAILED` with no message. `confluent-kafka` logs `SASL authentication error:` followed by nothing, and requests fail with `_TRANSPORT`. - If the client sets a SASL authorization identity, it must be the user name. Otherwise the broker answers `Authentication failed: SASL PLAIN authorization identity (authzid) must match authentication identity (authcid)`. - There's no re-authentication: a connection stays authenticated until it closes. ### mTLS With mTLS (5.5), the client authenticates with its own certificate on a TLS connection. Your platform team decides which CA to trust: one for the whole cluster, or up to two per tenant (a third is refused with "We cannot have more than 2 per tenant TLS certificates for protocol KAFKA."). The certificate's common name (CN) is looked up as a VAST user name, and that user is who the connection acts as for permissions. What happens to a connection without a usable certificate: | Client certificate | What happens | | --- | --- | | Signed by a CA the cluster doesn't trust | The TLS handshake fails with `unknown ca`. | | None | The TLS handshake completes, and the broker closes the connection at the first request. | | CN isn't a VAST user | The same: the broker closes the connection at the first request. | | Revoked in a revocation list your platform team uploaded for the CA | The TLS handshake fails with `certificate revoked`. | With no certificate, or a CN that isn't a user, `confluent-kafka` logs `Disconnected while requesting ApiVersion: might be caused by broker version < 0.10`. Revocation through CRL distribution points in certificates isn't supported. The platform team uploads a revocation list instead. mTLS uses TLS 1.2, like the other TLS connections, so leave TLS 1.2 enabled in your client. With mTLS, the broker views of several tenants can share one VIP pool. The cluster tells the tenants apart by the client certificate. ## Client settings These are `confluent-kafka` (librdkafka) settings. Add them to the configuration of every producer, consumer and admin client. Bootstrap addresses use the port of the connection kind: 9092 or 9093. Each configuration below was used to produce, consume and run admin calls against a view in that mode. No authentication, unencrypted: ``` conf = { "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"], # ":9092,..." "security.protocol": "PLAINTEXT", } ``` SASL/PLAIN over TLS (recommended for SASL): ``` conf = { "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"], # ":9093,..." "security.protocol": "SASL_SSL", "sasl.mechanism": "PLAIN", "sasl.username": os.environ["KAFKA_USERNAME"], "sasl.password": os.environ["KAFKA_PASSWORD"], "ssl.ca.location": "/path/to/kafka-ca.pem", } ``` SASL/PLAIN, unencrypted: as above, with `"security.protocol": "SASL_PLAINTEXT"`, port 9092 and no `ssl.ca.location`. TLS without client authentication: ``` conf = { "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"], # ":9093,..." "security.protocol": "SSL", "ssl.ca.location": "/path/to/kafka-ca.pem", } ``` mTLS: ``` conf = { "bootstrap.servers": os.environ["KAFKA_BOOTSTRAP"], # ":9093,..." "security.protocol": "SSL", "ssl.ca.location": "/path/to/kafka-ca.pem", # CA of the cluster's Kafka certificate "ssl.certificate.location": "/path/to/client.pem", # your certificate, CN = your VAST user "ssl.key.location": "/path/to/client.key", } ``` ## Permissions ### When permissions are checked Permissions are checked only when your platform team turns on **authorization** for the Kafka view. Authorization needs an authentication method on at least one connection kind, so that the cluster knows who the user is. The VAST CLI refuses it otherwise, with `At least one of the authentication mechanisms must be allowed in order to enable authorization for the KAFKA protocol`. - **Authorization off:** every client that can connect can do everything: produce, consume, and create or delete topics and groups. This includes users with no identity policy at all. - **Authorization on:** only operations that the user's identity policy explicitly allows are performed. An operation that no statement matches is refused, and so is every operation on a connection that hasn't authenticated, for example an unencrypted connection without authentication on a view that uses mTLS on TLS. A `Deny` statement overrides an `Allow`. The cluster evaluates the identity policies attached to the user and its groups. Bucket policies on the Kafka view's bucket don't apply to Kafka operations. ### Actions Kafka actions are written without a prefix, for example `"Action": "KafkaRead"`. Each row below was checked with authorization on, by granting and removing the action: | Action | Allows | | --- | --- | | `KafkaWrite` on a topic | Producing to the topic | | `KafkaRead` on a topic | Fetching from the topic | | `KafkaRead` on a consumer group | Joining the group, heartbeats, and committing and fetching its offsets | | `KafkaDescribe` on a topic | Seeing the topic in metadata, and looking up its offsets. Without it the topic is left out of metadata, so clients can't produce to it or consume from it: they get `TOPIC_AUTHORIZATION_FAILED`. | | `KafkaDescribe` on a consumer group | Listing and describing the group. Without it the group is left out of group lists. | | `KafkaCreate` on the cluster and on the new topic | Creating the topic. Both are needed. | | `KafkaDelete` on a topic or group | Deleting the topic, or the consumer group | | `KafkaDescribeConfigs` on the cluster, and `KafkaCreate` on the topic | Reading the topic's configuration (`describe_configs()`). Without `KafkaDescribeConfigs` you get `CLUSTER_AUTHORIZATION_FAILED`; without `KafkaCreate` on the topic, `TOPIC_AUTHORIZATION_FAILED`. | | `KafkaAlterConfigs` on the cluster and on the topic | Changing the topic's configuration (`alter_configs()`). Both are needed. | So a consumer typically needs `KafkaRead` and `KafkaDescribe` on its topics, and `KafkaRead` on its group. A producer needs `KafkaWrite` and `KafkaDescribe` on its topics. `Kafka*` matches every Kafka action. `s3:Kafka*` matches none of them. ### Resources The resources are: | Resource | Format | | --- | --- | | The Kafka view (cluster) | `vast:kafka:cluster:<bucket>` | | A topic | `vast:kafka:topic:<bucket>:<topic>` | | A consumer group | `vast:kafka:group:<bucket>:<group>` | `<bucket>` is the Kafka view's bucket. `"*"` matches every resource, and `*` inside a name works as a wildcard: `vast:kafka:topic:<bucket>:orders*` matches every topic whose name starts with `orders`. A resource that doesn't start with `vast:kafka:` (or the DataEngine prefix) is refused when the policy is saved, with `Resource VRN=… requires data engine or kafka prefix`. An example identity policy for a consumer of one topic, with its own group. With it, the user consumed and committed offsets, and producing was refused: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "ReadOrdersTopic", "Effect": "Allow", "Action": ["KafkaRead", "KafkaDescribe"], "Resource": "vast:kafka:topic::orders" }, { "Sid": "UseOwnGroup", "Effect": "Allow", "Action": "KafkaRead", "Resource": "vast:kafka:group::orders-reporting" } ] } ``` ### What a refused operation looks like | Refused | Client sees | | --- | --- | | Produce or fetch on a topic | `TOPIC_AUTHORIZATION_FAILED` for that topic. A consumer reports `Subscribed topic not available: <topic>: Broker: Topic authorization failed`. | | A consumer group operation | `GROUP_AUTHORIZATION_FAILED`, for example `JoinGroup failed: Broker: Group authorization failed` | | Creating a topic, or reading or changing configurations | `CLUSTER_AUTHORIZATION_FAILED` when the cluster action is missing, `TOPIC_AUTHORIZATION_FAILED` when the topic action is | | Deleting a topic | `TOPIC_AUTHORIZATION_FAILED` | ### No Kafka ACLs The Kafka ACL API isn't supported: identity policies take its place. A `DescribeAcls` request gets `SECURITY_DISABLED` ("No Authorizer is configured on the broker"), which `confluent-kafka`'s `describe_acls()` returns as an empty list. `create_acls()` and `delete_acls()` fail in the client with `_UNSUPPORTED_FEATURE`, and a client that sends those requests anyway has its connection closed. Tools that manage Kafka ACLs (such as `kafka-acls`) don't work. ## Reading topics as tables Reading a topic as a table through the VAST DataBase SDK, the Query Engine, Trino or Spark is a VAST DataBase request, not a Kafka one. It uses S3 access keys and the usual `s3:Tabular*` permissions on the Kafka view's bucket, not the Kafka actions. Blob expansion needs `s3:TabularCreateBlobExpansion`, `s3:TabularAlterBlobExpansion`, `s3:TabularDropBlobExpansion` and `s3:TabularGetBlobExpansion`. See [VAST DataBase security](/develop/docs/dev-database-operations-security) and [setup prerequisites](/develop/docs/dev-database-operations-setup-prerequisites#permissions). So one VAST user can need two credentials: a password (SASL/PLAIN) or certificate (mTLS) for Kafka, and an S3 key pair for SQL and the SDK. ## Auditing Producing, consuming and topic management aren't recorded by VAST protocol auditing, and aren't subject to Quality of Service limits. If you need an audit trail of who read or wrote a topic, keep it in your application. ## What to ask your platform team - Which connection kinds the view allows, and the port of each (9092, 9093). - The authentication method on each kind. - For TLS: the CA certificate that signed the cluster's Kafka certificate. - For SASL/PLAIN: a VAST local user and its password. - For mTLS: a client certificate with your VAST user name as its CN, its key, and confirmation that the CA is trusted for your tenant. - Whether authorization is on, and an identity policy with the actions above for your topics and groups. - For SQL and SDK reads: an S3 key pair and `s3:Tabular*` read access to the Kafka view's bucket. The procedures are in the KB: [user authentication](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-user-authentication-for-the-vast-event-broker-view), [mTLS authentication](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-mtls-authentication-for-the-vast-event-broker-view), [authorization](/docs-5-5/docs/publishing-events-to-vast-event-broker#configuring-user-authorization-for-the-vast-event-broker-view) and [Kafka permissions in an identity policy](/docs-5-5/docs/publishing-events-to-vast-event-broker#granting-kafka-permissions-in-an-identity-policy). ## See also - [Setup](/develop/docs/dev-event-broker-operations-setup): everything to ask for, and a connection check. - [Kafka compatibility](/develop/docs/dev-event-broker-reference-kafka-compatibility): supported and unsupported APIs. - [Troubleshooting](/develop/docs/dev-event-broker-troubleshooting-index): authentication failures and dropped connections. - [VAST DataBase security](/develop/docs/dev-database-operations-security): identities and policies for SQL and the SDK.