Memory Issues: linear growth in the memory when using daily indexing


(Pravendra Singh) #1

Hi all,

I have an ELK setup that uses per-day-indexing PREFIX-%{+YYYY.MM.dd}.

The memory consumed by the setup keeps growing with more indices, leading to a system restart with extra memory allocation. I observe a drop in the memory usage after the restart but eventually ending up in the same situation.

It's storing almost 5,00,000 events daily with 5 shards and a replica. Currently, I have allocated 32GB RAM to the system running ELK.

I am assuming this is because of the daily-indexing, and plan to switch to a monthly-indexing.

It will be helpful if you can review my observations and guide for the same.

Is there any rule of thumb to scale with more events?
OR
Is there anything else that I'm missing completely?


(Christian Dahlqvist) #2

Read this blog post for guidande on shard count and size.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.