Using version 6.8.1 at the moment, trying to wrap my head around indices and ILM and I'm completely corn-fusled.
Example:
Packetbeat.yml contains:
setup.template.name: "packetbeat-%{[beat.version]}"
setup.template.pattern: "packetbeat-%{[beat.version]}-*"
output.elasticsearch.index: "packetbeat-%{[beat.version]}-%{+yyyy.MM}"
And that creates a monthly index (though it includes "01" as the day).
I try adding:
output.elasticsearch.ilm.enabled: true
output.elasticsearch.ilm.pattern: "{now/M}-000001"
and in Kibana I edit the beats-default-policy to set max index size to 10GB and max age to 30 days (figuring to force a rollover ever 30 days or 10GB, whichever comes first) but the ILM doesn't roll it over to (for example) packetbeat-6.8.1-2019.06.01-000002 when the max size for the ILM policy is reached.
I used packetbeat in this example, but I have the same trouble with all the beats. Obviously I'm doing something wrong, but honestly, I have no idea what. I'm missing something, probably something really simple, and that's screwing me up and I've found nothing at all in the docs that heads me in the right direction.
The goal is a monthly index (ideally not including the day) with a serial indicating rollover because of size. (Daily indices from multiple beats over a 6 month period create a lot of sometimes tiny indices) I tossed in the day limit in the Kibana example above simply to try another condition to force a rollover because the size alone wasn't triggering one.
On light days we're down around 100K or less for daily indices, and I was hoping to use ILM to maintain a consistent size of around 5GB, but nothing I put in the size parameter seems to have any effect (and yes, when I ask Kibana to list the indices affected by that ILM, the index is listed among them). Instead of 4-5 indices in the last month, I see one huge one.