I want to use structured streaming for EventHub message consumption using pyspark in Azure Synapse and I was pointed to this project https://github.com/alexott/databricks-playground/tree/main/kafka-eventhubs-aad-auth. Could someone guide me if this could be modified to use certificate instead of secret authentication?, I tried to modify the ServicePrincipalCredentialsAuth class shown below to use ClientCredentialFactory.createFromCertificate but I am not able to make it work.

I want to use structured streaming for EventHub message consumption using pyspark in Azure Synapse, I found out this documentation https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-aad-authentication-to-connect-eventhubs.md that requires to create a callback function and load it into the spark pool as a package, has anyone done something similar for pyspark?