Relative Content

Tag Archive for apache-kafka-connectdebezium-connect

Kafka connect source rate control

I’m using the Debezium SQL Server Source Connector to get data from a table, put it into Kafka messages, and then consume it in a service. The problem is that the consuming process can be relatively slow, and the table is quite large, so I would like to control the rate of data production by the connector. I understand that I can manually monitor the consumer group lag of the service and, if the lag exceeds a certain threshold, I can turn off the Kafka Connect process, but I prefer a more automatic approach.