Writing a small paraquet dataframe into google cloud storage using spark 3.5.0 taking too long
We are using Spark onpremise to simply read a parquet file from GCS(google cloud storage) into the dataframe and write the dataframe into another folder in parquet format in GCS,using below code:
Writing a small paraquet dataframe into google cloud storage using spark 3.5.0 taking too long
We are using Spark onpremise to simply read a parquet file from GCS(google cloud storage) into the dataframe and write the dataframe into another folder in parquet format in GCS,using below code: