I setup MM2 to migrate the old data from the Kafka setup, which is very primitive consisting of one Zookeeper and one Kafka instances, to new KRaft cluster which has 3 controllers and 3 brokers. In the old setup, there are 44 topics and 2 of them have around 3.5m offsets. But the number of messages that is replicated to the new cluster is around 350k. I tried several configs but it didnt change the numbers. Other topics not fully replicated neither. One topic I’ve keeping my eye on is constantly replicating 8k message out of 50k in the old setup. I want know why the offsets not replicated alltogether. The whole idea was migrating the data but this kinda becomes a problem. This is the connector config;

I’m trying to carry the volume of the zookeeper cluster, which is a one kafka broker, to our new kafka cluster with KRaft. To do this I tried to setup a kafka connect and use mirror maker. I managed to start connect and create a connector between clusters and I also see that topics have been created in the new cluster but their old data is not there.

I’m trying to carry the volume of the zookeeper cluster, which is a one kafka broker, to our new kafka cluster with KRaft. To do this I tried to setup a kafka connect and use mirror maker. I managed to start connect and create a connector between clusters and I also see that topics have been created in the new cluster but their old data is not there.

I’m trying to carry the volume of the zookeeper cluster, which is a one kafka broker, to our new kafka cluster with KRaft. To do this I tried to setup a kafka connect and use mirror maker. I managed to start connect and create a connector between clusters and I also see that topics have been created in the new cluster but their old data is not there.

When I create a new topic in source cluster with few messages, the new topic will be replicated successfully to target cluster in few seconds. Then I create a consumer group in source, consume 1 message in source, this consumer group will also be replicated successfully to target cluster. Current offset in this new consumer group in both cluster will be 1 since I consumed 1 message.
Then I consume next message in source, the offset in consumer group in source cluster will be 2, but the offset in target cluster is still 1.

I am in the process of setting up an active-passive Kafka cluster and am using MirrorMaker 2 (MM2) for replicating topics between the source and destination clusters. My setup includes:

As a part of migration from one AWS MSK (v2.2.1) to another AWS MSK (v3.5.1), we are planning to make use of mirror maker 2 to replicate topics and topic events. However i was able to setup the replication fine with topics with messages, i had few edge cases where we wanted to replicate the topics with no events (topics with 0 messages) which was necessary for some of our services to mark as healthy. How to get this done? It is essential for us to create the topics with 0 events and this should be utmost done using mirror maker rather doing this manually as it would be tedious for our process.
Mirror maker version : kafka_2.12-3.5.0

I use mirrormaker 2 to sync kafka A-> B. I want to increase performance consunmer group synchrony by increasing partition of topic checkpoint. Is there any way to config the number partion topic checkpoint of mirrormaker2