---
title: "Topic configuration"
slug: "dev-event-broker-reference-topic-configuration"
description: "Explore how VAST Event Broker handles Kafka topic configs, including honoured, ignored, and rejected settings for optimal topic management."
updated: 2026-10-28T02:15:03Z
published: 2026-10-28T02:15:03Z
canonical: "docs.vastdata.com/dev-event-broker-reference-topic-configuration"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vastdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Topic configuration

*Applies to VAST Cluster 5.5.*

A topic has Kafka topic configs, as in Apache Kafka. VAST Event Broker honours some of them. It accepts and stores others but ignores them, and it rejects the rest. This page lists each config and what the broker does with it. For how to create topics and set configs, see [topics](/develop/docs/dev-event-broker-guides-topics).

## How the broker treats a config

| Status | Meaning |
| --- | --- |
| Honoured | The broker applies the config. |
| Stored and ignored | The broker checks the value's type, stores it and returns it from `describe_configs()`, but doesn't change its behaviour. A value of the wrong type is rejected with `INVALID_CONFIG`. |
| Rejected | `CreateTopics` or `AlterConfigs` fails with `INVALID_CONFIG`. |

Any config name the broker doesn't know is rejected.

## Honoured configs

| Config | What it does | Values | Default |
| --- | --- | --- | --- |
| `retention.ms` | Time-based retention. Records older than this are deleted in the background. | Milliseconds, or `-1` for no time limit. Don't rely on less than 6 hours: shorter values are accepted, but see the notes below. | 7 days (`604800000`) |
| `cleanup.policy` | `delete` removes records by age. `compact` keeps only the latest record for each key in a partition. | `delete`, `compact`, `delete,compact`, `compact,delete` (the VAST CLI accepts them too) | `delete` |
| `min.compaction.lag.ms` | The minimum time a record stays uncompacted. | Milliseconds, 0 or more | `0` |
| `delete.retention.ms` | How long a tombstone (a record with a key and a null value) is kept after compaction. | Milliseconds, 0 or more | 1 day (`86400000`) |
| `max.message.bytes` | The largest record batch a producer can send to the topic. A larger batch fails with `MESSAGE_TOO_LARGE`, which `confluent-kafka` reports as `MSG_SIZE_TOO_LARGE` ("Broker: Message size too large"). | Bytes. In the VAST Web UI this is "Max size", in megabytes. | `1048588` (about 1 MB) |
| `message.timestamp.type` | Which timestamp consumers get with each record. `CreateTime` returns the producer's timestamp; `LogAppendTime` returns the time the broker received the record. | `CreateTime`, `LogAppendTime` | `CreateTime` |
| `message.timestamp.before.max.ms` | With `CreateTime`, how far before the broker's time a record's timestamp may be. A record outside the window is rejected with `INVALID_TIMESTAMP` ("Broker: Invalid timestamp"). | Milliseconds, 0 or more | No limit (`9223372036854775807`) |
| `message.timestamp.after.max.ms` | With `CreateTime`, how far after the broker's time a record's timestamp may be. A record outside the window is rejected with `INVALID_TIMESTAMP`. | Milliseconds, 0 or more | No limit (`9223372036854775807`) |

The defaults are what `describe_configs()` returns for a new topic on VAST Cluster 5.5.0-SP6. You can see a topic's current values the same way.

### Notes on the honoured configs

- **Retention is approximate.** A background job deletes expired records, so they stay readable for a while after they pass `retention.ms`.  Don't rely on retention shorter than 6 hours: on a topic with `retention.ms` of 5 minutes, all its records were still readable 54 minutes after they were written.
- **Retention is based on the broker's timestamp,** the time the broker received the record, not the producer's `CreateTime`.
- **Retention never removes a partition's last record.**
- **Compaction runs in the background.** Duplicate keys may exist at any given time. With `min.compaction.lag.ms` and `delete.retention.ms` of 0, a compacted topic can keep only the latest record for each key about ten minutes after the writes, with a tombstoned key gone altogether.
- **Large messages and compaction.** If Max size is set above 5 MB, the topic isn't compacted and database queries on the topic aren't supported. The broker refuses to combine `compact` with a `max.message.bytes` above 5 MB, at topic creation or in `alter_configs()`, with `INVALID_CONFIG`: 6 MiB and 10 MiB were refused, 5 MiB accepted. It doesn't check the records already in the topic: a topic holding a 15 MB record could be switched to `compact` with a 4 MiB maximum.
- **Changes apply at once.** When you change retention or compaction with `alter_configs()`, the broker reschedules its background jobs for every partition straight away.
- **`validate_only`** is honoured by `CreateTopics` and `AlterConfigs`: nothing is changed. It checks config names and values, but not everything a real request checks: a partition count above the maximum, and `compact` with a `max.message.bytes` of 10 MiB, both pass `validate_only` and fail for real.

### Where you can set them

You can set all the honoured configs with the Kafka Admin API (`create_topics()` and `alter_configs()`). The platform team can also set them in the VAST Web UI, the VAST CLI (`topic create` and `topic modify`) or the VMS REST API, except that the CLI has no option for the maximum message size. Use the non-incremental `alter_configs()`: the broker doesn't support `incremental_alter_configs()`, which fails with `_UNSUPPORTED_FEATURE`. See [Kafka compatibility](/develop/docs/dev-event-broker-reference-kafka-compatibility#unsupported-features).

Unlike Apache Kafka, `alter_configs()` changes only the configs you send: the others keep their values. For example, setting only `retention.ms` on a compacted topic leaves `cleanup.policy=compact` in place.

The VAST CLI offers only the honoured configs, and so do the Web UI and REST API. The ignored configs below can be set only through the Kafka Admin API.

## Stored and ignored configs

The broker accepts these configs, type-checks their values and returns what you set from `describe_configs()`, but they have no effect.

| Config | Why it has no effect |
| --- | --- |
| `compression.type`, `compression.gzip.level`, `compression.lz4.level`, `compression.zstd.level` | The broker rejects compressed batches from producers, whatever the topic says. VAST compresses the stored data itself. Setting `compression.type` on the topic doesn't let producers compress. |
| `min.insync.replicas`, `unclean.leader.election.enable`, `follower.replication.throttled.replicas`, `leader.replication.throttled.replicas` | There's no Kafka-level replication. Durability comes from VAST storage. |
| `segment.bytes`, `segment.ms`, `segment.jitter.ms`, `segment.index.bytes`, `index.interval.bytes`, `preallocate`, `file.delete.delay.ms`, `flush.messages`, `flush.ms` | Topics are VAST DataBase tables, not log segment files. |
| `min.cleanable.dirty.ratio` | Compaction is scheduled by `min.compaction.lag.ms`. |
| `max.compaction.lag.ms` | Accepted. Whether it affects compaction isn't confirmed. |
| `message.timestamp.difference.max.ms` | Use `message.timestamp.before.max.ms` and `message.timestamp.after.max.ms`. |
| `message.format.version`, `message.downconversion.enable` | The broker uses message format v2 only. `message.format.version` accepts only `3.0-IV1`, the value `describe_configs()` returns; other values such as `3.0` or `2.8` are rejected with `INVALID_CONFIG`. |
| `local.retention.ms`, `local.retention.bytes`, `remote.storage.enable`, `remote.log.copy.disable`, `remote.log.delete.on.disable` | There's no tiered storage. |

`describe_configs()` returns every config in this table along with the honoured ones, 35 in all, filled in with defaults, and it reports none of them as read-only. Don't read a returned value as a sign that the broker uses it.

## Rejected configs

| Config | What happens | What to do |
| --- | --- | --- |
| `retention.bytes` | `INVALID_CONFIG` ("Broker: Configuration is invalid"), in `create_topics()` and in `alter_configs()`. There's no size-based retention. | Leave it out. Use `retention.ms`. |
| Any other config name | `INVALID_CONFIG` | Leave it out. |
| Broker configs (resource type `BROKER`) | `describe_configs()` returns none, and a change is rejected with `INVALID_REQUEST`. | None. There are no broker configs to change. |

Some tools and frameworks set `retention.bytes` or other configs when they create topics. If topic creation fails with `INVALID_CONFIG`, check which configs the tool sends.

## The partition count

The partition count isn't a config, but it's set at the same time. Pass it explicitly when you create a topic: a count of `-1` (the broker default, and the default in `confluent-kafka`'s `NewTopic`) or `0` is rejected with `INVALID_PARTITIONS`. You can't change it after the topic is created. For the maximum, see [limits](/develop/docs/dev-event-broker-reference-limits).

## See also

- [Topics](/develop/docs/dev-event-broker-guides-topics): creating topics and changing their configs.
- [Kafka compatibility](/develop/docs/dev-event-broker-reference-kafka-compatibility)
- [Limits](/develop/docs/dev-event-broker-reference-limits)
- [Troubleshooting](/develop/docs/dev-event-broker-troubleshooting-index)
- [Publishing events to VAST Event Broker](/docs-5-5/docs/publishing-events-to-vast-event-broker), "Managing Event Topics"
- VAST CLI [topic create](/docs-5-5/docs/topic-create) and [topic modify](/docs-5-5/docs/topic-modify)