How to filter dataframe by column from different dataframe?
I want to filter dataframe by column with Strings from different dataframe.
set difference of a column in spark dataframe
I have a dataframe like this :
Scala – Convert Map to Dataframe where Keys re the Column Titles
I wish to create a dataframe by using a map such that the keys of the map are the column titles, and the values of the map are the data itself. In python and pyspark, this can be done quite easily in numerous ways- typically with one line of code, even-, but in Scala, I’m having serious trouble.
It is possible to use spark Dataframe/Dataset api with accumulators?
I read and filter data, need to count how each filter operation affects result.
Is it possible to somehow mixin spark accumulators while using Dataframe/Dataset api?
Scala : create dataframe column from array where the array size is variable
I have a variable like
val activityId = “activity_” + activityNum + “_id”
the variable activityNum is incremented from a loop (1,2,3,…)
How to compare last year’s data in Scala?
I’m trying to compare the data from the same month but previous year and add previous year data to a new column ‘qty_ly’. As well as sum the last 3months of qty_cy for each product.