Index lifecycle error - 2 index templates matching

Elastic Stack Beats
,
1

Dear experts,
here is my filebeat setup:

# Set the prefix used in the index lifecycle write alias name. The default alias
# name is 'filebeat-%{[agent.version]}'.
setup.ilm.rollover_alias: 'filebeat-%{[agent.version]}-nginxws'
# Set the rollover index pattern. The default is "%{now/d}-000001".
setup.ilm.pattern: "{now/d{yyyy.MM.dd}}-000001"
# Set the lifecycle policy name. The default policy name is
# 'beatname'.
setup.ilm.policy_name: "datastream_policy_30d"

Things are going pretty good: rollover mechanism create new files like this one: filebeat-7.10.2-nginxws-2021.04.01-000004
However, from time to time, I have this kind of annoying errors:
illegal_argument_exception: index.lifecycle.rollover_alias [filebeat] does not point to index [filebeat-7.10.2-nginxws-2021.04.01-000004]

I would say it is not error because I setup extra alias filebeat-%{[agent.version]}-nginxws which is actually pointing to filebeat-7.10.2-nginxws-2021.04.01-000004

{
  "filebeat-7.10.2-nginxws-2021.04.01-000004" : {
    "aliases" : {
      "filebeat-7.10.2-nginxws" : {
        "is_write_index" : true
      }
    }
  }
}

I guess there is something like a competition between alias filebeat and filebeat-7.10.2-nginxws:

{
  "filebeat-7.10.2-nginxws" : {
    "order" : 0,
    "index_patterns" : [
      "filebeat-7.10.2-nginxws-*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "datastream_policy_30d",
          "rollover_alias" : "filebeat-7.10.2-nginxws"
        },

{
  "filebeat" : {
    "order" : 1,
    "index_patterns" : [
      "filebeat-*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "datastream_policy_30d",
          "rollover_alias" : "filebeat"
        },

I try to play with the order value to make some progress but no way ....

Any idea to avoid this error ?

Thanks

2

Here is the workaround which avoid index.lifecycle.rollover_alias error. I updated the index_patterns value from filebeat template this way:

{
  "filebeat" : {
    "order" : -1,
    "index_patterns" : [
      "toto-*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "datastream_policy_30d",
          "rollover_alias" : "filebeat"
        },

As a consequence, filebeat-7.10.2-nginxws template does match filebeat-7.10.2-nginxws-* indices as a single match.

I am a little bit surprised that Elastic does not properly manage this case. I will dig into component templates as an alternative to legacy index templates which are going to be deprecated any way.

I'll keep you informed.
Cheers

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.