There is a specific task I have to work on. Basically, there are many servers from where I have to collect some time series data (positions if that matters), and store them in a remote server (there will be a process in this remote server running which collects the data and store it in a DB). These positions are server specific, so I will most likely store them in different databases, just like before the collection. Currently it is stored in PostgreSQL, but the problem is that in that remote server I need to store them for years. There is quite a lot of data, the initial estimate is around 400-600 TB for 7 years (and this is compressed size, with TimescaleDB, so in case I can’t use that It can be 10x bigger based on some compression estimate…). This is another reason why I will store the data in separate DBs, because PostgreSQL says to be able to store around 32 TB of data per database (although TimescaleDB partitions by default so I guess It would be solved with that).

There would be an estimated write of around 80K records / sec. Read would be very rare, It would be basically just stored for archiving, but sometimes It is possible someone will do some query (not very likely). BUT, the data retention and consistency is a must, so there should be no data loss (for this there will be some RAID/NAS storage).

I read that for many writes, few reads NoSQL or Document based databases are better, but for consistency people usually suggest RDMBS. What would be the better option in this case?

Thank you for your answers!

As mentioned above, currently postgres is used, but I’m not sure if that is the best solution.