When I try to run a pyspark step on my EMR cluster I get an error Caused by: java.lang.ClassNotFoundException: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. My understanding from AWS documentation is that the EMR file system should already be installed on my EMR cluster? I also tried referencing my .py file in s3 using s3a instead, and get a similar error saying the S3a file system can’t be found.

When I try to run a pyspark step on my EMR cluster I get an error Caused by: java.lang.ClassNotFoundException: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. My understanding from AWS documentation is that the EMR file system should already be installed on my EMR cluster? I also tried referencing my .py file in s3 using s3a instead, and get a similar error saying the S3a file system can’t be found.

When I try to run a pyspark step on my EMR cluster I get an error Caused by: java.lang.ClassNotFoundException: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. My understanding from AWS documentation is that the EMR file system should already be installed on my EMR cluster? I also tried referencing my .py file in s3 using s3a instead, and get a similar error saying the S3a file system can’t be found.

When I try to run a pyspark step on my EMR cluster I get an error Caused by: java.lang.ClassNotFoundException: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. My understanding from AWS documentation is that the EMR file system should already be installed on my EMR cluster? I also tried referencing my .py file in s3 using s3a instead, and get a similar error saying the S3a file system can’t be found.

When I try to run a pyspark step on my EMR cluster I get an error Caused by: java.lang.ClassNotFoundException: Class com.amazon.ws.emr.hadoop.fs.EmrFileSystem not found. My understanding from AWS documentation is that the EMR file system should already be installed on my EMR cluster? I also tried referencing my .py file in s3 using s3a instead, and get a similar error saying the S3a file system can’t be found.

I have a rather strange issue in a managed pyspark environment that’s hosted on EMR 6.10.1