My application polls records from a MSK Kafka cluster. The application maintains offset of each partition and hence has disabled autocommit. It actually never commits offset as it persists offset to internal data store.

Suppose two processes are executing two separate Kafka Publish Transactions of 10 messages each, parallelly. After both commit, will the messages from both transactions be interleaved in the partition? or Kafka guarantees that messages of the same transaction are grouped together in the partition? The different “producer workers” have different transactional IDs

As I know Kafka consumer is not thread safe. So how can asyncCommit be asynchronous if it uses one main application thread?

I am using kafka UI as a visual interface for viewing topics. I wanted to check if my service produced correct number of messages to the topic. When i used live mode in given topic i was able to confirm that 500 messages were produced (as expected) but when i go to overview of the topic. The total number always increased more than 500. Sometimes more than twice. I even checked on confluent and the total number of messages is same as kafka UI presents.

I have a kafka cluster and there are some jvm’s collected to read messages from this kafka cluster. I have closed some of these jvms to never open them again and there is lag accumulation in the consumergroups of these closed jvms in kafka, what should I do for this? Is it necessary to delete these consumer groups and will the number of lags be an issue?

In Kafka, I know by using read_commited, we only get already committed messages.
Does this mean when we poll messages, do we always receive entire committed messages?

Let’s say one of your consumers is unresponsive and it exceeds its connect timeout because it got stuck doing something, there will be a rebalance and the partitions that were assigned to the stuck consumer will be assigned to other consumers and the message that was never acknowledged will also be given to another consumer.

Can someone tell me how to delink consumer group from kafka topics. I want to delete kafka topic but that topic is linked with a consumer group. Is there any way to remove kafka topic from consumer group usi g kafka cli?

This topic has multiple produce, some messages cannot be processed, Most of the messages are being consumed normally.

I am polling the messages from the kafka topic and it start rebalancing for around 30 mins and get stable after it. In kafka logs I am a getting