i am looking some suggestions for setting up the ILM for my data/streaming/beats . i have below senerios to accomplished and keep running my platforms.
Everyday the indices are generating with abc.yyy.mm.dd format, without any data size limits of indices within the logstash mapping and patterns, the problem here how can i setup my ILM to avoid the flood of data sizes (regular pattern is around 1gb data but suddenly if data comes to 50gb)
can we setup ILM policy or rollover configuration in the Logstash pipelines with a indices size limits?
is there any better approach to keep my platform up and running with limited to the disk spaces/cluster storages.
i am using the ELK 8.3 with FB agents in multiple hosts.
Thank you in advance for your early responses and suggestions.