To quote the docs:
Writing to different indices: best practices
You cannot use dynamic variable substitution when ilm_enabled is true and when using ilm_rollover_alias.
So what I think I want to do is make each new batch of documents be ingested in a new index, where the name starts the same. Eg:
logstash-2021.12.10
logstash-2021.12.11
logstash-2021.12.12
The example output config of
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
worked at first, creating "logstash-2021.12.10-000001". Im not 100% sure where the "-000001" came from, but that does not bug me too much.
but this no longer works. It creates the index of a literal "%{[@metadata][beat]}-%{[@metadata][version]}-".
Notice that the date part is entirely missing.
I found the able about ilm, and attempted to turn it off with:
POST /_ilm/stop
So... first: dont I need dynamic index names for ilm? And what setting is preventing this?