7.9 ILM Policy - Rollover never happens

[Mitchell (Mitch B)]

Hello,

If I have an index named config and a corresponding index pattern config*, it is unclear to me, after several attempts, how to follow the steps in this tutorial

https://www.elastic.co/guide/en/elasticsearch/reference/7.9/getting-started-index-lifecycle-management.html

To get an ILM policy to begin monitoring the index named config, and when the rollover conditions are met, begin automatically creating new indices named config-000001, config-000002, etc.

In the tutorial, I have chosen to follow the steps that do not use data streams. Here is the output of ...

GET config*/_ilm/explain

    "indices" : {
        "config" : {
          "index" : "config",
          "managed" : true,
          "policy" : "config-ilm-policy",
          "lifecycle_date_millis" : 1611541975204,
          "age" : "52.72m",
          "phase" : "hot",
          "phase_time_millis" : 1611543849160,
          "action" : "rollover",
          "action_time_millis" : 1611543309825,
          "step" : "check-rollover-ready",
          "step_time_millis" : 1611543849160,
          "is_auto_retryable_error" : true,
          "failed_step_retry_count" : 4,
          "phase_execution" : {
            "policy" : "config-ilm-policy",
            "phase_definition" : {
              "min_age" : "0ms",
              "actions" : {
                "rollover" : {
                  "max_size" : "5mb",
                  "max_age" : "1d",
                  "max_docs" : 900
                },
                "set_priority" : {
                  "priority" : 100
                }
              }
            },
            "version" : 1,
            "modified_date_in_millis" : 1611542322493
          }
        }

The rollover settings are just for testing. I did run the following successfully as well:

    PUT config-000001
    {
      "aliases": {
        "conf": {
          "is_write_index": true
        }
      }
    }

    PUT /_cluster/settings
    {
      "persistent": {
        "indices.lifecycle.poll_interval": "1m"
      }
    }

However, only my config index is being written to. The ILM rollover settings are never enforced and the conf-00001 index exists but never gets documents added to it. Both index and index.lifecycle.rollover_alias aliases are "conf". I received an error when attempting to make an alias "config", the same name as the initial index.

[afharo (Alejandro Fernandez Haro)]

Hi @Mitchell,

It seems to me like you're having some issues when trying to apply your ILM policy to your config* indices.

First of all, I want to tackle this comment:

I received an error when attempting to make an alias "config", the same name as the initial index.

The reason for this is that an index and an alias can't have the same name. You might want to Reindex your existing index config to config_1 or config-00001. Then you can delete the original config index, and create the alias.

A second comment to your tests: based on the output of GET config*/_ilm/explain, I can see these properties:

          "is_auto_retryable_error" : true,
          "failed_step_retry_count" : 4,

It looks like it's failing to apply the ILM policy. The logs in ES should provide the reason for the errors. My initial guess is that it's related to some issues with the aliases (i.e.: they should only point to 1 index as "is_write_index": true), but the logs will certainly help finding out