Hardware Recommendations for Shipping IIS Logs


(Machina X (Machina)) #1

Hi there.

I'm going to deploy new IIS Log monitoring system with elasticsearch + Hadoop.

The Elasticsearch Cluster would be for storing Microsoft IIS Log. And Hadoop would backup the expired elasticsearch indices(over 180 days).

IIS Log Size is 5.3 TB per a month. So I think that 32 TB SSD Volume Size is appropriate for me because I should store total 3 months logs.

_But, Following some articles and some experiments which is done by myself, If over 90 % of total disk is used, System automatically alert low disk and lock some kibana monitoring indices. _ To avoid those lack of available disk situation,

Now I'm carefully calculating required disk spec, so I want to know how much percentage of spare disk would be need ?