Applies to VAST Cluster 5.5.
This page lists the limits of VAST Event Broker in VAST Cluster 5.5. Topics are VAST DataBase tables, so the VAST DataBase limits also apply where this page doesn't say otherwise.
Topics and partitions
| Limit |
Value |
| Partitions per topic |
Up to 20,000. A topic with 20,001 is refused with INVALID_PARTITIONS. Topics with more than 1,000 partitions work. |
| Partitions per Kafka view |
Up to 200,000 |
| Changing the partition count |
Not possible after the topic is created |
| Topics per cluster |
Limited by the maximum number of tables per database and by the total number of partitions |
| Automatic topic creation |
Not supported |
| Retention |
Default 7 days. Minimum 6 hours. -1 means no time limit. The Kafka API and the VAST CLI accept shorter values, but a topic with 5-minute retention still held all its records 54 minutes later: don't rely on retention below 6 hours. |
| Event queries based on the topic partition |
Not supported |
Messages
| Limit |
Value |
| Message size |
Set per topic by max.message.bytes ("Max size" in the VAST Web UI). Its default is about 1 MB: a 1.1 MB record is refused with MSG_SIZE_TOO_LARGE ("Broker: Message size too large"), a 1,000,000-byte one is accepted. With a larger setting, larger records work: 15 MB on a topic set to 20 MiB. |
| Key and value size |
No separate limit below the message size: a 200 KB key with a 200 KB value is accepted on a topic with the default settings. |
| Maximum cell size for Kafka topics |
1 GB. For other VAST DataBase tables it's 5 MB; see the VAST DataBase limits. |
| Topics with a maximum message size above 5 MB |
Not compacted: compaction with a maximum of 6 MiB is refused with INVALID_CONFIG, with 5 MiB it's accepted. Database queries on the topic aren't supported: an SDK query on a topic with a 20 MiB maximum failed with QueryDataInternalError ("Query failed due to server error: error_code=20"). |
| Compression over the wire |
Not supported. Producers must send uncompressed batches. |
Consumer groups
| Limit |
Value |
| Consumer groups per Kafka view |
Plan for up to 256. A view accepted 278 groups with committed offsets, all returned by list_consumer_groups(). |
| Replication of consumer groups |
Not supported. Committed offsets aren't replicated. |
Cooperative rebalancing, READ_UNCOMMITTED, client rack awareness, fetch sessions. Static membership (group.instance.id) is accepted but ignored |
Not supported. See Kafka compatibility for what clients see. |
Kafka views and connections
| Limit |
Value |
| VIP pools per Kafka view |
1: the VAST CLI refuses more with "Too many VIP pools provided. Max allowed number is 1.". The pool can be modified, but not replaced by another pool, after the view is created. |
| VIPs in the pool |
At least one VIP per CNode |
| Sharing a VIP pool between Kafka views |
Allowed only if every Kafka view on the pool uses mTLS. A second view on a pool is refused otherwise, also within one tenant: "Only MTLS authentication is supported because the Kafka VIP is used by other views". Between tenants, the views tell tenants apart by the client certificate. |
| Kafka views per cluster |
Limited by the maximum number of views and VIP pools on the cluster |
| Adding the Kafka protocol to an existing view |
Not allowed |
| Removing the Kafka protocol from a view that has topics |
Not allowed: "Deleting KAFKA protocol from view with existing topics is not allowed" |
| Kafka TLS certificates |
1 per cluster. It needs one IP-address SAN entry for every VIP in every Kafka VIP pool on the cluster. |
| TLS version |
TLS 1.2, for TLS and mTLS connections. A client limited to TLS 1.3 fails the handshake. |
| mTLS revocation |
Revocation through CRL distribution points isn't supported. A revocation list uploaded for the CA is enforced. |
| CA certificates for mTLS per tenant |
Up to 2. A third is refused. |
| SASL/PLAIN users |
VAST local users only. Active Directory and LDAP users aren't supported. |
| Protocol auditing and QoS |
Publishing, consuming and topic management aren't covered |
Blob expansion
| Limit |
Value |
| Expansion tables per source column |
1 |
| Format |
JSON only |
| Source column |
value only |
| Target table |
Must exist before you configure the expansion. Otherwise create_blob_expansion() fails with NotFound ("failed to get target table for blob expansion"). |
| Renaming or removing target-table columns that the expansion uses |
Not allowed |
| Renaming the source topic |
Not allowed. Topics can't be renamed at all. |
See blob expansion.
Data protection and other features
| Limit |
Value |
| Replicated topics |
Read-only at the destination peer until failover |
| Fast restore of a protected path that contains a Kafka view |
Not allowed |
| Global Access |
Kafka views aren't supported |
| S3 event notifications to a third-party broker |
Up to five broker hosts |
See also