Starting Spark Shell with Amazon EMR 7.2 and Apache Hudi. Permission denied: user=ec2-user
[ec2-user@ip-172-31-46-115 ~]$ spark-shell –jars /usr/lib/hudi/hudi-spark-bundle.jar –conf “spark.serializer=org.apache.spark.serializer.KryoSerializer” –conf “spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog” –conf “spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension” Aug 13, 2024 8:36:31 PM org.apache.spark.launcher.Log4jHotPatchOption staticJavaAgentOption WARNING: spark.log4jHotPatch.enabled is set to true, but /usr/share/log4j-cve-2021-44228-hotpatch/jdk17/Log4jHotPatchFat.jar does not exist at the configured location Setting default log level to “WARN”. To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 24/08/13 20:36:41 ERROR SparkContext: Error initializing […]