Does ILM work in Logstash with custom index names?

Hi there, we are currently using a single, generic "logstash-*" ElasticSearch output which works quite well with the default ILM defaults of 30 days or 50GB and the automatic alias naming convention of logstash-2020.04.03-000001, logstash-2020.04.13-000002, logstash-2020.04.24-000003, etc.

We are currently adding different data types via additional pipelines in a single logstash process but would like to keep those different data types for different amounts of time. I was thinking of creating multiple indexes with different names using a mutate like:

mutate { add_field => { "[@metadata][target_index]" => "prod-%{+YYYY.MM.dd}" } }

but then I saw the comment here: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#_writing_to_different_indices_best_practices

You cannot use dynamic variable substitution when ilm_enabled is true and when using ilm_rollover_alias .

How can I enable custom logstash indices and have the associated ILM policies like: logstash-custom1-2020.04.24-000003?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.