EMR Serverless SparkSession builder error: ClassNotFoundException issues
I am trying to create a job in EMR Studio to run in an EMR Serverless application. It’s a relatively basic script to use PySpark to read some Athena tables, do some joins, create an output dataframe and write back to S3 as parquets.