Kafka
Kafka consumer lag keeps growing: read the group, then find out why
Lag that only grows means the group is not committing. Read kafka-consumer-groups --describe and --state, spot a rebalance storm, and fix max.poll.records.
Consumer lag, ISR, min.insync.replicas and offset resets: operating Apache Kafka from the command line.
3 postsRSS feed
Lag that only grows means the group is not committing. Read kafka-consumer-groups --describe and --state, spot a rebalance storm, and fix max.poll.records.
Producers with acks=all get NotEnoughReplicasException when the ISR is below min.insync.replicas. Find the missing brokers and fix them, not the setting.
kafka-consumer-groups --reset-offsets --to-datetime: stop the group, mind the milliseconds and UTC, dry-run, export a backup, then execute, and undo it.