I have accidentally performed partition increase to __consumer_offets
topic in Kafka (Was version 2.4 now it’s 3.6.1)
Now when I list the consumer groups using Kafka CLI, I get a list of consumers which I’m unable to delete
List command
kafka-consumer-groups --bootstrap-server kafka:9092 --list | grep -i queuing.production.57397fa8-2e72-4274-9cbe-cd42f4d63ed7
Delete command
kafka-consumer-groups --bootstrap-server kafka:9092 --delete --group queuing.production.57397fa8-2e72-4274-9cbe-cd42f4d63ed7
Error: Deletion of some consumer groups failed:
* Group 'queuing.production.57397fa8-2e72-4274-9cbe-cd42f4d63ed7' could not be deleted due to: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.GroupIdNotFoundException: The group id does not exist.
So after this incident we got an advice to change all of our consumer groups names so that new consumer groups will be created and we won’t loose data and have incosistancy, We done so and everything was back to normal.
But
We still have tons of consumer groups that we are unable to remove from the list probably because of this __consumer_offsets
partition increase.
This is a Production cluster so shutting it down is not an option.
We would like to remove them without any interruption to the producers and consumers of this cluster. Is it possible? or are we stuck with them forever?