I have an ELK setup that uses per-day-indexing
The memory consumed by the setup keeps growing with more indices, leading to a system restart with extra memory allocation. I observe a drop in the memory usage after the restart but eventually ending up in the same situation.
It's storing almost
5,00,000 events daily with
5 shards and
a replica. Currently, I have allocated
32GB RAM to the system running ELK.
I am assuming this is because of the daily-indexing, and plan to switch to a monthly-indexing.
It will be helpful if you can review my observations and guide for the same.
Is there any rule of thumb to scale with more events?
Is there anything else that I'm missing completely?