Unable to create a new index applying ILM policy

Elastic Stack Logstash
1

Hi all,

We have set up the elasticsearch output plugin in Logstash 7.5 to write to a index name and we see that the index is created correctly in ElasticSearch 7.5 if it does not exist. Now we would like to test how can we configure logstash to create the index with a policiy that includes rollover, so that the first index would be created as "ilmindex -000001" with "ilmindex" as alias, and "ilmpolicy" applied. We have tested several options in logstash.conf but with no luck, the index is always created as "ilmindex" with no alias.

Our elasticsearch output section in logstash.conf looks like this:

output {
    elasticsearch {
         hosts => "http://elasticsearch:9200"
         manage_template => "false"
         index => "ilmindex"
         ilm_enabled => "true"
         ilm_policy => "ilmpolicy"
    }
}

"ilmindex" index template is set up this way:

"ilmindex" : {
     "order" : 0,
    "version" : 1,
    "index_patterns" : [
        "*"
    ],
    "settings" : {
        "index" : {
            "lifecycle" : {
                "name" : "ilmpolicy",
                "rollover_alias" : "ilmindex"
            },
            "number_of_shards" : "1",
            "refresh_interval" : "5s"
        }
    },
    "mappings" : { },
    "aliases" : { }
}

And this is how we have defined "ilmpolicy" index lifecycle policy:

"ilmpolicy" : {
    "version" : 1,
    "modified_date" : "2019-12-12T09:08:27.285Z",
    "policy" : {
        "phases" : {
            "hot" : {
                "min_age" : "0ms",
                "actions" : {
                    "rollover" : {
                        "max_size" : "1gb",
                        "max_age" : "1d"
                    },
                    "set_priority" : {
                        "priority" : 100
                    }
                }
            },
            "delete" : {
                "min_age" : "2d",
                "actions" : {
                    "delete" : { }
                }
            }
        }
    }
}

Something must be missing in our configuration files but after testing many configruation options form logstash.conf we have not been able to create the index automatically with ILM enabled. Any help would be much appreciated.

Thanks in advance.

Jose Antonio

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.