Relative Content

Tag Archive for apache-kafkakafka-consumer-apikafka-partition

Golang Segmentio Getting half partitions while Consumer Group Migration

I am using github.com/segmentio/kafka-go lib.
My service is hosted on Kubernetes cluster and my use case is, I have one consumer group as C1, which is consuming messages from topic T1 which has 70 partitions. I have fetched the latest committed offsets for topic T1 by the consumer C1 for all the 70 partitions. Now I have created a new, dynamic, consumer group for example as C2. So, as per my approach, I have committed offsets for all the 70 partitions and now I will fetch the assigned partitions for the particular pod of the new consumer group i.e. C2 and the seek and set the offsets of the assigned partitions as per old consumer C1.
So I am using group.Next() for getting the generation and from that I am fetching the assigned partitions for the particular pod of the C2. But here I am getting only half of partitions i.e. 35 partitions.