Relative Content

Tag Archive for c++parallel-processingioparquetapache-arrow

recordbatchreader failed when reading parquet file

I tried to use arrow::recoredbatchreader to read multiple rowgroups from a parquet file in parallelism. I use GetRecordBatchReader to acquire recordbatchreader. However, I noticed that when the number of task exceeded the number of cores, the reading would stop at RETURN_NOT_OK(ReadNext(&batch));. The recordbatchreader only works when the number of tasks is less than the number of cores.