I need an official documentation or any official way suggesting a rough log consumption estimate Gb/day for list of data sources --like i have list which include
so any documentation or any other official way that suggest that take the average eps for data source = x & average byte size = y so the formula becomes
Don't estimate it, measure it. Or help the client to measure it.
e.g. the firewall log volume will be completely dependent on what events you/they choose to log, how verbose, etc.
And, in elasticsearch context, the peak volume, and its distribution over time, is almost as important as any daily average. I worked on a solution that averaged 1k EPS for 99% of the day, but 100k EPS for the other 1% (about 15 mins/day). During which period is it most important to know quickly if things are working well ?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.