Im having issues flattening a nested dataframe in spark which I have solved using a custom function but I am wondering if there is a better way to go about this.