Illegal_argument_exception: index.lifecycle.rollover_alias [cisco-switch] does not point to index [cisco-switch-2022.07.27]

Hello,

after trying to figure out some stuff about ILM i need some help now.
I have about 400 Cisco-Devices sending their logs to a syslog-ng server from where they are send to Elasticsearch via Filebeat / Logstash.
The logs on the syslogserver are rotated every day at 0:00 UTC and elasticsearch indexing the logs as
cisco-switch-2022-07.26
cisco-switch-2022.07.27
and so on.

I have followed this tutorial to create a ILM : Tutorial: Automate rollover with ILM | Elasticsearch Guide [8.11] | Elastic and usesd the 90-days-default ILM

I did the folowing steps:

  1. Create the index template
PUT _index_template/cisco-switch
{
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "90-days-default",
          "rollover_alias": "cisco-switch"
        },
        "number_of_shards": "1",
        "number_of_replicas": "0"
      }
    }
  },
  "index_patterns": [
    "cisco-switch-*"
  ],
  "composed_of": []
}
  1. create initial index
PUT cisco-switch-000001
{
  "aliases": {
    "cisco-switch": {
      "is_write_index": true
    }
  }
}
  1. Start Filebeat and Logstash to start loading the logs into elasticsearch

But it doesnt seems to work at all cause im getting the folowing error

Index lifecycle error

illegal_argument_exception: index.lifecycle.rollover_alias [cisco-switch] does not point to index [cisco-switch-2022.07.27]

All i want is that indices older than 90ß days get deleted from the index .

Any help would be appreciated.

Hello,

I'm not a professional, I have discovered those ILM last week and came across this problem too.

According to the documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-error-handling.html#_index_lifecycle_rollover_alias_x_does_not_point_to_index_y

You should check your aliases with _cat/aliases.. There I noticed that my first index didn't have the alias (despite the fact I use the "is_write_index": true too).

One solution was to run:

  • template (as you did)
  • create the index (as you did)
  • run PUTcisco-switch-000001/_alias/cisco-switch to link the alias to the index

It's just for the first index, then it's automatic.
I hope it will works for you too!

Can you share your ILM policy as well please.

@bianca6
thank you very much.
put _cat/aliases shows

.transform-notifications-read           .transform-notifications-000002                         - - - -
.kibana-event-log-8.3.2                 .kibana-event-log-8.3.2-000001                          - - - true
.preview.alerts-security.alerts-default .internal.preview.alerts-security.alerts-default-000059 - - - false
.kibana_task_manager_8.2.2              .kibana_task_manager_8.2.2_001                          - - - -
.lists-default                          .lists-default-000001                                   - - - true
.preview.alerts-security.alerts-default .internal.preview.alerts-security.alerts-default-000060 - - - true
.kibana_task_manager                    .kibana_task_manager_8.3.2_001                          - - - -
.kibana_task_manager_8.3.2              .kibana_task_manager_8.3.2_001                          - - - -
.items-default                          .items-default-000001                                   - - - true
.kibana_8.2.2                           .kibana_8.2.2_001                                       - - - -
.security                               .security-7                                             - - - -
cisco-switch                            cisco-switch-000001                                     - - - -
.kibana                                 .kibana_8.3.2_001                                       - - - -
.kibana_8.3.2                           .kibana_8.3.2_001                                       - - - -
.kibana-event-log-8.2.2                 .kibana-event-log-8.2.2-000002                          - - - false
.kibana-event-log-8.2.2                 .kibana-event-log-8.2.2-000001                          - - - false
cisco-asa                               cisco-asa-000001                                        - - - -
.kibana-event-log-8.2.2                 .kibana-event-log-8.2.2-000003                          - - - true

but cisco_switch-2022.07.26, cisco-switch-2022.07.27 and cisco-switch2022.07.28 are not listed there, same for cisco-asa-2022.07.xx

somehow the template seems to not add the aliases to the new created indices and i keep getting the error.

@warkolm im using the default ILM Policy 90-days-default that came with ELK-Stack without any changes to it, but here it is:

PUT _ilm/policy/90-days-default
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "30d",
            "max_primary_shard_size": "50gb"
          }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "shrink": {
            "number_of_shards": 1
          },
          "forcemerge": {
            "max_num_segments": 1
          }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {}
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    },
    "_meta": {
      "managed": true,
      "description": "built-in ILM policy using the hot, warm, and cold phases with a retention of 90 days"
    }
  }
}

If you are sending data to time-based indices with date in the name you can not use rollover as part of ILM. The whole point of rollover is to have a single alias that you write to and let Elasticsearch roll over backing indices automatically based on size and age. This relies on you giving up control of exactly which data that go into which backing index.

You therefore have 2 options. The first is to use rollover and require you to write to the cisco-switch alias instead of index names with date in the names.

If you instead prefer to write to indices in the form cisco-switch-2022-07.26 you should remove rollover from the ILM policy.

Hello @Christian_Dahlqvist

You mean something like this should delete my indices when they become older than 90 days?

PUT _ilm/policy/cisco-90-days
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "set_priority": {
            "priority": 100
          }
        },
        "min_age": "0ms"
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Yes, something like that.

thank you :slight_smile: I will try

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.