Logstash Indices Rollover to Warm

We have been having issues getting our Logstash data to rollover from hot to warm after 30 days on our Elastic Cloud server. Before we change any settings, we would like to make sure we understand what we are doing.

Currently the output in the Logstash pipeline is set as ilm_enabled => auto and we had created an Index Lifecycle Policy named company-logstash-policy in the Stack Management set to rollover from hot to warm at 50GBs or 30 days. We are getting several errors including:

illegal_argument_exception: index.lifecycle.rollover_alias [company-logstash-alias] does not point to index [company-logstash-2021.11.11]
illegal_argument_exception: rollover target [company-logstash-alias] does not point to a write index

From reading Elasticsearch output plugin as well as a discussion posts, should we leave the policy, modify the output to look similar to this snippet, and reindex all current Logstash indices?

output {
  elasticsearch {
    cloud_id => "<ID>"
    ssl => true
    user => "<USERNAME>"
    password => "<PASSWORD>"
    index => "company-logstash-%{+YYYY.MM.dd}"
    ilm_enabled => true
    ilm_rollover_alias => "company-logstash-alias"
    ilm_pattern => "000001"
    ilm_policy => "company-logstash-policy"

As result we should have hot indices like:




And then after 30 days they will rollover to warm indices like:

company-logstash-alias -000001

company-logstash-alias -000002

company-logstash-alias -000003

Are we understanding this correctly or will this lead to further issues in our stack? Is this issue caused by something completely different?

Thank you

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.