Relative Content

Tag Archive for pythonpandasdaskpython-xarraydask-dataframe

Making conversion of data array to Dataframe faster with x-array and Dask

I am trying to convert an x array data array into a pandas dataframe at a faster time. without using Dask I was able to convert a data array with 17 variables of 9GB each into a pandas dataframe in around 19seconds. After implementing dask with 7 workers and 10 threads per worker (wasn’t getting any better with increased workers or thread.) I was only about to make it about 3 seconds faster (16 seconds). Is there a way I could make it faster or is there something I am missing. Below is my code: