I have got a POC up and running locally without any compression or optimization and its sitting at 1.3x of the source file size. The next step is to move into building out a dev cluster with 7tb of data. I am trying to come up with an estimate as to how much storage I will need. Replication isn't necessary. Just the bare minimum estimate...Ive seen estimate ranges from 1.1x - 2.5x.
Thanks
Estimates will vary since they depend on things like index mappings and the input data itself. If you have a 1.3x estimate for actual data I'd use that.
I wasn't sure if that was going to be indicative once we scaled the data set up. Either way, it should be a good starting point, thanks!
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.