I am trying to split a large file (>20GB) in chunks to upload them.
This is my method:
public List<byte[]> chunkFile() throws IOException {
File file = new File(backupPath);
List<byte[]> chunks = new ArrayList<>();
try (FileInputStream fis = new FileInputStream(file)) {
byte[] buffer = new byte[1024 * 1024 * 10];
int bytesRead;
while ((bytesRead = fis.read(buffer)) > 0) {
byte[] chunk = new byte[bytesRead];
System.arraycopy(buffer, 0, chunk, 0, bytesRead);
chunks.add(chunk);
}
return chunks;
}
}
It works well to small files (500mb), but when I try with a larger file I am facing this error:
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler dispatch failed: java.lang.OutOfMemoryError: Java heap space] with root cause
java.lang.OutOfMemoryError: Java heap space
I need to split the file in chunks to be able to upload to s3 using multi part file.