Relative Content

Tag Archive for scalaapache-sparkpysparkamazon-emr

Spark-Scala vs Pyspark Dag is different?

I am converting pyspark job to Scala and jobs executes in emr. The parameter and data and code is same. However I see the run time is different and so also the dag getting created is different. Here I am adding the data read part from UI. If you see the number of outputrows task in the UI.