The problem I have recently upgraded my apps to run on Spark3.5.1+YARN3.3.6, and observing frequent failures saying “Authorized committer”. The apps run PySpark and I observe the error always happens in the output stage (while writing to S3). Has anyone had a similar experience after upgrading to the recent version of Spark? I am suspecting […]