Relative Content

Tag Archive for apache-kafkaconfluent-platform

Filter$Value class Not Found while Kafka Single Message Transforms implementation

We’re customizing docker through docker file, executing them in kubernetes, trying to ingest data from Kafka topics, followed by few transformations. We could receive data from kafka topics when no transformations applied. We’re trying Kafka Single Message Tranforms to filter specific data from topic but throws us below error

Kafka max.request.size VS compression.type

I am testing a program to verify if compression helps to reduce the size of the topic message. My sample topic has config ‘max.message.bytes=1024000’ which is ~1MB and on the producer side config I have set the same value to ‘max.request.size’ and then I attempted to send across a string that is of size 1573015 which is ~1.5MB and this throws below error which is expected.

Kafka max.request.size VS compression.type

I am testing a program to verify if compression helps to reduce the size of the topic message. My sample topic has config ‘max.message.bytes=1024000’ which is ~1MB and on the producer side config I have set the same value to ‘max.request.size’ and then I attempted to send across a string that is of size 1573015 which is ~1.5MB and this throws below error which is expected.

Kafka max.request.size VS compression.type

I am testing a program to verify if compression helps to reduce the size of the topic message. My sample topic has config ‘max.message.bytes=1024000’ which is ~1MB and on the producer side config I have set the same value to ‘max.request.size’ and then I attempted to send across a string that is of size 1573015 which is ~1.5MB and this throws below error which is expected.

Can’t migrate from Zookeeper to KRaft

I’m having trouble with this situation. First I tried to migrate the volume which is in zookeeper to new kraft cluster with MirrorMaker2 but I couldn’t manage it. I posted that as a question as well. Now I tried to follow the migration steps that is described in Confluent’s Docs. But I couldn’t manage this also.

Zookeeper to KRaft post-migration question: is __cluster_metadata actually a topic?

We recently started migrating our test clusters from Zookeeper to KRaft, all running Confluent Kafka on prem. In KRaft, cluster’s metadata is stored in __cluster_metadata topic (see: https://developer.confluent.io/courses/architecture/control-plane/#kraft-cluster-metadata) . What I noticed after the first cluster was migrated is that I am not able to describe this topic using kafka-topics command, it just does not exist: