kafka-topics.sh: create, delete, describe, or change a topic
kafka-topics.sh [options]
Options you will use
--bootstrap-server <String: server to connect to>- REQUIRED: the Kafka broker(s) to connect to, host:port[,host:port]. Give more than one: the first may be the broker that is down.
--command-config <String: config file>- a properties file with client configs for the admin client (TLS, SASL...).
--list- list all available topics.
--describe- list details for the given topics: TopicId, PartitionCount, ReplicationFactor, Configs (overrides only), then per partition Leader, Replicas, Isr, Elr, LastKnownElr.
--create- create a new topic.
--alter- alter the number of partitions (only up).
--delete- delete a topic.
--topic <String: topic>- the topic to create, alter, describe or delete (a regex for --describe/--delete/--alter).
--partitions <Integer: # of partitions>- the number of partitions for the topic being created or altered.
--replication-factor <Integer>- the replication factor for each partition (default: the broker's default.replication.factor).
--replica-assignment <String>- a manual assignment: partitions separated by commas, the brokers of each by colons (1:2:3,2:3:1).
--config <String: name=value>- a topic config override for the topic being created.
--under-replicated-partitions- only partitions whose ISR is smaller than the replica list.
--under-min-isr-partitions- only partitions whose ISR is below min.insync.replicas.
--at-min-isr-partitions- only partitions whose ISR equals min.insync.replicas.
--unavailable-partitions- only partitions without a leader.
--exclude-internal- leave out internal topics such as __consumer_offsets.
--topics-with-overrides- with --describe: only topics that have config overrides.
--if-not-exists- with --create: no error if the topic exists.
--version- display Kafka version.
Examples
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --topic ordersleaders, replicas and ISR of every partition
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --under-replicated-partitionsonly the problems (empty = healthy)
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --create --topic refunds --partitions 6 --replication-factor 3 --config retention.ms=1209600000a topic with an override
Gotchas
- --alter can only add partitions, and adding them moves keys to other partitions.
- With auto.create.topics.enable=false (the lab) a producer to a missing topic waits max.block.ms and fails instead of creating it.
- Configs: in --describe lists overrides only (and the cluster-wide min.insync.replicas default); everything else is inherited.
Taught in
- 28.1 Kafka for the on-call engineer: the log, the cluster, the lab
- 28.5 Topics, partitions and replicas: reading kafka-topics --describe
- 28.7 Durability: acks, the ISR, min.insync.replicas and ELR
- 28.15 Storage: segments, retention, compaction and full disks
- 28.19 Moving partitions: reassignment, throttles and preferred leaders
- 28.23 Running brokers: rolling restarts, the JVM and the metrics that page you
On the blog
Try kafka-topics.sh in a real terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.