Topics, partitions, replicas
A topic is a name; the work happens in its partitions. Each partition is an ordered log on disk, and Kafka only guarantees order inside one partition. Partitions are also the unit of parallelism: a consumer group can have at most one active consumer per partition, so a topic with 6 partitions can be read by at most 6 consumers of one group at the same time.
Each partition is replicated to replication factor brokers. One replica is the leader: all produce and fetch requests for that partition go to it. The others are followers: they fetch from the leader, exactly like a consumer, to stay in sync. Different partitions have their leaders on different brokers, so the load is spread.
Reading kafka-topics.sh --describe
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --topic orders
Topic: orders TopicId: hbZw5PYaJOpDrtsWAGGWFw PartitionCount: 6 ReplicationFactor: 3 Configs: min.insync.replicas=2,segment.bytes=1073741824
Topic: orders Partition: 0 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Elr: LastKnownElr:
Topic: orders Partition: 1 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Elr: LastKnownElr:
Topic: orders Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Elr: LastKnownElr:
Topic: orders Partition: 3 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Elr: LastKnownElr:
Topic: orders Partition: 4 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Elr: LastKnownElr:
Topic: orders Partition: 5 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Elr: LastKnownElr:
The first line is the topic:
- TopicId - a UUID given at creation. A topic deleted and re-created with the same name gets a new one; clients that cached the old id notice.
- PartitionCount, ReplicationFactor.
- Configs - only the settings that are not Kafka's built-in defaults.
segment.bytes=1073741824is there becauselog.segment.bytesis set in the brokers'server.properties(a static broker setting is not "default" even when it has the default value);min.insync.replicas=2because the cluster has it as a dynamic default (lesson 4 and 9 explain both).
Then one line per partition:
- Leader - the broker id of the leader, or
nonewhen no replica can lead: the partition is offline, nobody can write to it or read from it. - Replicas - the assigned brokers. The first one is the preferred leader: when everything is healthy, it should be the leader (lesson 10).
- Isr - the in-sync replicas, the ones fully caught up with the leader. Healthy = same set as Replicas.
- Elr / LastKnownElr - the eligible leader replicas (Kafka 4, lesson 4). Empty when healthy.
Creating a topic
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --create --topic invoices --partitions 6 --replication-factor 3
Created topic invoices.
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --topic invoices
Topic: invoices TopicId: 1QSzIzeUKaFWUhqj3GEJPg PartitionCount: 6 ReplicationFactor: 3 Configs: min.insync.replicas=2,segment.bytes=1073741824
Topic: invoices Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Elr: LastKnownElr:
Topic: invoices Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Elr: LastKnownElr:
Topic: invoices Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Elr: LastKnownElr:
Topic: invoices Partition: 3 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3 Elr: LastKnownElr:
Topic: invoices Partition: 4 Leader: 2 Replicas: 2,3,1 Isr: 2,3,1 Elr: LastKnownElr:
Topic: invoices Partition: 5 Leader: 3 Replicas: 3,1,2 Isr: 3,1,2 Elr: LastKnownElr:
The controller spreads the replicas round-robin over the brokers, starting at a different broker for each topic, so leaders end up balanced. Without --partitions and --replication-factor the topic gets the cluster's num.partitions and default.replication.factor - both 1 out of the box. In KRaft the controller creates the topic and fills them in, so the lab sets them (3 and 3) in the controllers' server.properties; a broker that has them in its file logs a WARN at start that 5.0 will ignore them there. A replication factor larger than the number of brokers is refused:
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --create --topic big --replication-factor 4
Error while executing topic command : Unable to replicate the partition 4 time(s): The target replication factor of 4 cannot be reached because only 3 broker(s) are registered or some brokers have all their log directories cordoned.
[2026-09-22 20:00:12,800] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Unable to replicate the partition 4 time(s): The target replication factor of 4 cannot be reached because only 3 broker(s) are registered or some brokers have all their log directories cordoned.
(org.apache.kafka.tools.TopicCommand)
Note what goes where: the "Error while executing topic command" line goes to standard output, the ERROR log line with the exception to standard error. That ERROR line is the tool's log4j2 output (config/tools-log4j2.yaml, WARN and above, on stderr). Exceptions that come back from the brokers carry no stack trace, which is why the closing (org.apache.kafka.tools.TopicCommand) sits alone on the next line.
A name with . or _ gets a warning, because both turn into the same character in metric names:
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --create --topic payment_events.v2 --partitions 3
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic payment_events.v2.
Changing a topic
You can add partitions, never remove them:
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --alter --topic invoices --partitions 8
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --alter --topic invoices --partitions 4
Error while executing topic command : The topic invoices currently has 8 partition(s); 4 would not be an increase.
[2026-09-22 20:00:15,800] ERROR org.apache.kafka.common.errors.InvalidPartitionsException: The topic invoices currently has 8 partition(s); 4 would not be an increase.
(org.apache.kafka.tools.TopicCommand)
Adding partitions has a cost the help text spells out: records with a key go to murmur2(key) % number-of-partitions, so after the change the same key lands in a different partition, and per-key ordering across the change is gone. Plan the partition count up front. Topic settings (retention and the like) are not changed here but with kafka-configs.sh (lesson 9); --alter --config is refused with a message that says so.
Deleting is instant and prints nothing on success:
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --delete --topic payment_events.v2
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --list
__consumer_offsets
invoices
notifications
orders
payments
There is no undo. In production, auto.create.topics.enable=false (this lab sets it) stops a typo in a producer's config from creating a new topic silently - with the default true, the first write to ordres creates ordres with one partition and one replica.
Everything at once
--describe without --topic describes every topic, including the 50 partitions of __consumer_offsets. --exclude-internal leaves the internal topics out, and four filters only print the partitions that need attention - you will use them in the next lesson:
| Flag | Prints partitions where |
|---|---|
--under-replicated-partitions | ISR smaller than the replica set |
--under-min-isr-partitions | ISR smaller than min.insync.replicas (or no leader) |
--at-min-isr-partitions | ISR exactly min.insync.replicas: one more failure and writes stop |
--unavailable-partitions | no leader, or the leader is not alive |
$ kafka-topics.sh --bootstrap-server kafka-1:9092 --describe --under-replicated-partitions
Empty output is the good answer.
In an interview: "Ordering is per partition, parallelism is per partition, and you can only add partitions - which moves keys. RF 3 puts every partition on three brokers; the first replica in the list is the preferred leader."