Sizing question

Elastic Stack Elasticsearch
1

Hello,

We are storing firewall logs from our networking infrastructure.

The biggest user is the fortigate one with this biggest seen daily usage:

  • 135 GB
  • 173 332 370 documents

But the average is around 60 GB with a standard deviation of 30 GB

We're going to buy a new cluster with these specs:

  • 5 nodes with each:
    -- 96 GB RAM (the more the better for FS Cache)
    -- 30 GB heap each (near the recommended maximum)
    -- 10000 Write IOPS capacity
    -- 4 TB storage
    -- 2 CPUS 12 Cores

Other indices will be much smaller...

Buf, with thes fact at hand, is our setup seems to be ok?
Or our servers are too much dense in storage capacity and should go with more, less beefy nodes?

Thanks

2

How many days of logs are you planning on keeping around?

3

90 days retention

4

90 days of data

5

That seems sane based on what I've seen.

6