Relative Content

Tag Archive for apache-kafkaapache-kafka-streams

Kafka producer send record to topic partition catch TimeoutException: Expiring 1 record(s)

In a real environment, flink is used to connect to kafka as the producer of Kafka. Now, the “TimeoutException” exception message appears when sending data in the environment: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for topic-test-1 due to 60024 ms has passed since batch creation plus linger time
I observed the network IO and disk IO (network: 10000mbps) of the kafka server, and there is still a lot of idle time. All broker nodes in the kafka cluster are in normal state
Producer configuration:

Runtime reconfigure a Kafka Streams filter

How do you reconfigure a filter in a Kafka Streams app at runtime (without stopping and restarting the app)? The configuration for what to filter is ideally found in a Kafka configuration topic. The data is actually compacted (KTable) so what I’m trying to achieve is a rewind and reprocessing of all data on filter update as the output topic is also compacted and provides the state of the app after joining a few data tables and applying a filter.