Step-by-step instructions for the work you'll do most. Start with Manage topics, Produce events and Consume events; read Query topics as tables, Expand JSON events into columns and Publish S3 bucket events to a topic when you use those features.
- Manage topics: Create, inspect, change and delete topics from a Kafka admin client, including partitions, retention and compaction, and what the broker rejects.
- Produce events: Configure a confluent-kafka producer for Event Broker: required settings, acks, key partitioning, headers, batching and the errors it returns.
- Consume events: Read topics with confluent-kafka consumers: consumer groups, offset commits, rebalancing, direct partition reads, seeking, isolation and group deletion.
- Query topics as tables: Read Kafka-produced events as VAST DataBase tables with the Python SDK, the Query Engine, Trino or Spark, without a consumer.
- Expand JSON events into columns: Parse JSON event values into typed table columns as they're produced, then change or remove the expansion with the Python SDK.
- Publish S3 bucket events to a topic: Send S3 object create, tag and delete events to a topic with a bucket notification, and consume them with at-least-once delivery.