I have just find out spark structured streaming do not commit offset to kafka but use its internal checkpoint system and that there is no way to visualize its consumption lag in typical kafka UI

• https://community.databricks.com/t5/data-engineering/can-we-commit-offset-in-spark-structured-streaming-in-databricks/td-p/71900#:~:text=Spark%20Structured%20Streaming%20manages%20which,a%20spark%20structured%20streaming%20query ?

Lag being important in stream processing, I can’t imagine that the community did not come up with workaround to help with consumer lag tracking, but so far i could not find any out of the box solutions.

In any case, as i do not want to reinvent the will, I won’t if anyone can share solution either out of the box or custom that people typically uses for this ?