Storage Requirements for 7 TB dev cluster

(Andrew) #1

I have got a POC up and running locally without any compression or optimization and its sitting at 1.3x of the source file size. The next step is to move into building out a dev cluster with 7tb of data. I am trying to come up with an estimate as to how much storage I will need. Replication isn't necessary. Just the bare minimum estimate...Ive seen estimate ranges from 1.1x - 2.5x.


(Magnus B├Ąck) #2

Estimates will vary since they depend on things like index mappings and the input data itself. If you have a 1.3x estimate for actual data I'd use that.

(Andrew) #3

I wasn't sure if that was going to be indicative once we scaled the data set up. Either way, it should be a good starting point, thanks!

(system) #4