The title says it all. An iterable dataset with a multi-worker dataloader yields more batches than it should (seems that each worker yields all the batches separately). Here is an MWE:

I have an ordered dataset(shuffle=False) that is categorised into “bins”. I shall present an example on smaller scale that helps to clarify. Let’s say the size of dataset is 60 with bins of sizes 10,20,30. I want to train my model in the order of bins. (first with 10 then 20 and 30). I want my DataLoader to get data in batch_sizes of 8. In this case, after getting the first 8 datapoints, I don’t want to get the 2 remaining from bin-1 and get 6 from next one. What I want is to get only 2 and in the next iteration, get the 8 from bin-2. In short, I want to complete training in one bin first before moving to other. Also if batch_size happens to be greater than bin size, I want to get data in solely one bin before moving to next.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.

I want to utilize my GPU by adjusting the workers number, but I have a problem with the number of workers > 0.