OnCallReadyStart learning

Runtimes and data Lesson 28.13 · Kafka Operations · 12 min read

Resetting offsets safely

In plain words

A consumer group's committed offsets are bookmarks. Moving the bookmarks back makes the group read again from there; moving them forward skips pages that nobody will ever read. You can only move a team's bookmarks while the team is out of the room - otherwise they would put their own bookmarks back the moment you leave, and your work would be lost.

Why you move a group's offsets

The committed offsets are the group's memory. Moving them is how you:

  • replay: a bug in the consumer wrote wrong invoices for the last two hours; after the fix, go back two hours and process again;
  • skip: a poison record (one that makes the consumer crash every time) or a flood of junk nobody needs; jump past it;
  • recover: someone else moved the offsets by mistake; put them back.

All of it is kafka-consumer-groups.sh --reset-offsets. It is also the command that causes the most damage in Kafka operations, because it does exactly what it is told.

The rules

  1. The group must be inactive. While members are connected, their own commits would overwrite yours, so the tool refuses:
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --to-earliest --topic notifications --execute

Error: Assignments can only be reset if the group 'notifier' is inactive, but the current state is Stable.

GROUP           TOPIC           PARTITION  NEW-OFFSET

Stop the application first (all its instances, everywhere), check the group is Empty, reset, start it again.

  1. Without --execute nothing happens. The default is a dry run that prints the plan; 4.3 warns that version 5.0 will require --dry-run or --execute explicitly.
  1. Scope it. --topic notifications (all partitions), --topic notifications:0,2 (two partitions), or --all-topics (every topic the group has offsets for). One of them is required.
$ sudo systemctl stop notifier
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --describe --group notifier --state

Consumer group 'notifier' has no active members.

GROUP           COORDINATOR (ID)          ASSIGNMENT-STRATEGY  STATE                #MEMBERS
notifier        kafka-3:9092  (3)         -                    Empty                0
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --to-earliest --topic notifications
WARN: No action will be performed as the --execute option is missing. In version 5.0, this command will require either --dry-run or --execute to be specified. You should add the --dry-run option explicitly if you are scripting this command and want to keep the current default behavior without prompting.

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          0
notifier        notifications   1          0
notifier        notifications   2          0

The scenarios

OptionNew offset
--to-earliestthe first offset still in the log
--to-latestthe end: skip everything waiting
--to-offset NN, clamped to the log's range
--shift-by Nthe current committed offset + N (negative = back)
--to-datetime 2026-10-21T08:00:00.000the first offset at or after that time
--by-duration PT2Hthe first offset at or after now minus the duration (ISO-8601)
--to-currentthe committed offset (useful with --export)
--from-file plan.csvwhat a CSV says, per partition

--to-datetime has a trap. The format is YYYY-MM-DDThh:mm:ss.sss - the milliseconds are required, and without a zone it means UTC:

$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications --to-datetime 2026-09-22T19:00:00 --dry-run

Error: Executing consumer group command failed due to java.text.ParseException: Unparseable date: "2026-09-22T19:00:00Z"
java.lang.RuntimeException: java.text.ParseException: Unparseable date: "2026-09-22T19:00:00Z"
	at org.apache.kafka.tools.OffsetsUtils.resetToDateTime(OffsetsUtils.java:365)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand$ConsumerGroupService.prepareOffsetsToReset(ConsumerGroupCommand.java:984)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand$ConsumerGroupService.resetOffsetsForInactiveGroup(ConsumerGroupCommand.java:672)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand$ConsumerGroupService.lambda$resetOffsets$0(ConsumerGroupCommand.java:649)
	at java.base/java.util.HashMap.forEach(HashMap.java:1429)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand$ConsumerGroupService.resetOffsets(ConsumerGroupCommand.java:643)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand.run(ConsumerGroupCommand.java:129)
	at org.apache.kafka.tools.consumer.group.ConsumerGroupCommand.main(ConsumerGroupCommand.java:109)
Caused by: java.text.ParseException: Unparseable date: "2026-09-22T19:00:00Z"
	at java.base/java.text.DateFormat.parse(DateFormat.java:399)
	at org.apache.kafka.common.utils.Utils.getDateTime(Utils.java:1559)
	at org.apache.kafka.tools.OffsetsUtils.resetToDateTime(OffsetsUtils.java:354)
	... 7 more

Unparseable date is the tool parsing 2026-09-22T19:00:00Z with yyyy-MM-dd'T'HH:mm:ss.SSSXXX and failing. With the milliseconds:

$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications --to-datetime 2026-09-22T19:00:00.000 --dry-run

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          503988
notifier        notifications   1          503988
notifier        notifications   2          503988
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications --by-duration PT30M --dry-run

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          511269
notifier        notifications   1          511269
notifier        notifications   2          511269

A time before the oldest record gives the oldest offset; a time in the future gives the end. --to-offset and --shift-by beyond the log are clamped, with a WARN on stderr:

$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications:0 --to-offset 999999999 --dry-run
[2026-09-22 20:00:21,150] WARN New offset (999999999) is higher than latest offset for topic partition notifications-0. Value will be set to 518400 (org.apache.kafka.tools.OffsetsUtils)

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          518400

Doing it

Before you --execute, save where the group is now - --to-current --export writes the current positions as CSV, and --from-file puts them back if the reset was wrong:

$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications --to-current --export --dry-run > /tmp/notifier-before.csv
$ cat /tmp/notifier-before.csv
notifications,0,518400
notifications,1,518400
notifications,2,518400
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --topic notifications --shift-by -50 --execute

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          518350
notifier        notifications   1          518350
notifier        notifications   2          518350
$ kafka-consumer-groups.sh --bootstrap-server kafka-1:9092 --group notifier --reset-offsets --from-file /tmp/notifier-before.csv --execute

GROUP           TOPIC           PARTITION  NEW-OFFSET
notifier        notifications   0          518400
notifier        notifications   1          518400
notifier        notifications   2          518400
$ sudo systemctl start notifier

The CSV has topic,partition,offset per line (with one group; with several, the group comes first).

What replaying means downstream

Kafka gives no exactly-once delivery to the outside world: the notifier sends e-mails, the billing service writes invoices. Going back two hours means two hours of e-mails again, unless the consumer is idempotent (it remembers what it already did). Before a replay, ask the owners of the consumer what duplicates do to them. Skipping forward is the opposite risk: records nobody will ever process.

Committed offsets also expire: a group that is Empty for longer than offsets.retention.minutes (7 days) loses them, and when it starts again auto.offset.reset decides where it begins - the end, by default, silently skipping everything in between.

In an interview: "Stop every consumer of the group, dry-run the reset, export the current offsets, then execute. --to-datetime needs milliseconds and is UTC. A replay means duplicates downstream unless the consumer is idempotent."

Why it helps

Replaying after a bug, skipping a poison record and recovering from someone else's mistake are all offset resets. The tool does exactly what it is told, so the same command that fixes an incident can cause one. Knowing the rules - group inactive, dry run first, milliseconds in the date, export before execute - keeps a routine reset from becoming a duplicate-email incident.

On the blog

Commands in this lesson

kafka-consumer-groups.sh systemctl cat

FAQ

Why does the reset say the group must be inactive?

Active members commit their own positions all the time and would overwrite the new offsets. The tool refuses while the group has members, so you stop every instance of the application first and check that the state is Empty.

Does --reset-offsets change anything without --execute?

No. Without --execute it is a dry run that only prints the plan. Kafka 4.3 also prints a warning that version 5.0 will require --dry-run or --execute to be given explicitly, so scripts should already say which one they mean.

Why does --to-datetime fail with Unparseable date?

The format must be YYYY-MM-DDThh:mm:ss.SSS: the milliseconds are required. Without a zone the time is UTC. The tool appends Z and parses with a pattern that needs the milliseconds, so 2026-09-22T19:00:00 fails and 2026-09-22T19:00:00.000 works.

How do I undo a reset?

Export the positions before you change them: --to-current --export writes topic,partition,offset lines as CSV. --from-file with that file and --execute puts the group back exactly where it was, partition by partition, even hours later, as long as the records are still in the log.

What happens to offsets of a group that is unused for a long time?

They expire after offsets.retention.minutes (7 days) of the group being Empty. When the application starts again, auto.offset.reset decides where it begins - by default the end of the log, silently skipping everything in between, with no error at all.

In an interview Mid

How do you replay a topic for a consumer group from a point in time?

Stop every consumer of the group until --describe --state says Empty, export the current offsets with --reset-offsets --to-current --export, dry-run --reset-offsets --to-datetime with the time in UTC and the milliseconds, then run it with --execute and start the consumers. A replay means duplicates downstream unless the consumer is idempotent, so check with its owners first.

Also asked: Why must the group be inactive for a reset? · How do you undo a wrong reset? · What happens when a group's offsets expire?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.