I appreciate if I could get advice with number of indices.
I have 3 elasticsearch nodes with below spec for each node.
64 GB (30 GB heap)
48TB (RAID 1+0)
Our requirement is
- 60GB/day , with avg 500 Bytes per event.
- 40 types of servers and network devices
- Each logs should hold about year
- I want to split the logs into 60 indices per day with 2 primary shard + 1 replica
- Since each logs are different context resulting in different fields so each logs will be separate into 1 indice
- The total indices will count up to 7,300 indices per node in a year.
- Each indice will contain about 2Million docs.
However, reality is, since each indice will consume memory just by opening post . Is there any good way to keep all the indice open ? Perhaps, index alias will be a good idea in my caes?