Spark window shuffle phase
I am trying to better understand the behaviour of Spark’s window functions.
I find it very strange that there is no shuffle phase(exchange) if you repartition the dataframe on just one of the columns in the partitionBy clause.
I would expect it to do an exchange for the hash of all the columns in the stage, instead I just see a sort in the physical plan.