# Indexes not being rotated/removed using the Index Lifecycle Management

**URL:** <https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [June 11, 2020, 7:45pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777 "2020-06-11T19:45:58Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jbalaguero](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\
**Post date:** [June 11, 2020, 7:45pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777/1 "2020-06-11T19:45:58Z")

</div>

Hello,

I’m trying to use the Index Lifecycle Management in Elastic 7.7, but I’m not able to rotate indexes every 1 minute and delete indexes older than 3 minutes.

This is my policy:

[root@elasticmysql ~]# curl -GET [http://elasticmysql:9200/\_ilm/policy/ventus\_policy\_25?pretty=true](http://elasticmysql:9200/_ilm/policy/ventus_policy_25?pretty=true)

```auto
{
  "ventus_policy_25" : {
    "version" : 1,
    "modified_date" : "2020-06-11T19:02:39.913Z",
    "policy" : {
      "phases" : {
        "hot" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1m"
            }
          }
        },
        "delete" : {
          "min_age" : "3m",
          "actions" : {
            "delete" : { }
          }
        }
      }
    }
  }
}

```

This is my index template:

[root@elasticmysql ~]# curl -GET [http://elasticmysql:9200/\_template/pattern-25?pretty=true](http://elasticmysql:9200/_template/pattern-25?pretty=true)

```auto
{
  "pattern-25" : {
    "order" : 0,
    "index_patterns" : [
      "25_*"
    ],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "ventus_policy_25",
          "rollover_alias" : "25"
        },
        "refresh_interval" : "10000ms",
        "number_of_shards" : "1",
        "number_of_replicas" : "0"
      }
    },
    "mappings" : {
      "dynamic" : "strict",
      "_source" : {
        "enabled" : true
      },
      "properties" : {
        "creationTime" : {
          "index" : "true",
          "store" : "false",
          "type" : "long",
          "doc_values" : false
        },
        "expirationTime" : {
          "index" : "true",
          "store" : "false",
          "type" : "long",
          "doc_values" : true
        },
        "hotelCodeList" : {
          "index" : "true",
          "store" : "false",
          "type" : "keyword",
          "doc_values" : false
        },
        "cacheId" : {
          "index" : "false",
          "store" : "false",
          "type" : "keyword",
          "doc_values" : false
        }
      }
    },
    "aliases" : { }
  }
}

```

And this is my bootstrap index:

[root@elasticmysql ~]# curl -GET [http://elasticmysql:9200/25\_001?pretty=true](http://elasticmysql:9200/25_001?pretty=true)

```auto
{
  "25_001" : {
    "aliases" : {
      "25" : {
        "is_write_index" : true
      }
    },
    "mappings" : {
      "dynamic" : "strict",
      "properties" : {
        "cacheId" : {
          "type" : "keyword",
          "index" : false,
          "doc_values" : false
        },
        "creationTime" : {
          "type" : "long",
          "doc_values" : false
        },
        "expirationTime" : {
          "type" : "long"
        },
        "hotelCodeList" : {
          "type" : "keyword",
          "doc_values" : false
        }
      }
    },
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "ventus_policy_25",
          "rollover_alias" : "25"
        },
        "refresh_interval" : "10000ms",
        "number_of_shards" : "1",
        "provided_name" : "25_001",
        "creation_date" : "1591902160104",
        "number_of_replicas" : "0",
        "uuid" : "5bQidXKaSve5sg6b8UfC_A",
        "version" : {
          "created" : "7070099"
        }
      }
    }
  }
}

```

When I perform the explain after 1 minute I expect to find “25\_001” and “25\_002”, but I only see “25\_001”:

[root@elasticmysql ~]# curl -GET [http://elasticmysql:9200/25\_\*/\_ilm/explain?pretty=true](http://elasticmysql:9200/25_*/_ilm/explain?pretty=true)

```auto
{
  "indices" : {
    "25_001" : {
      "index" : "25_001",
      "managed" : true,
      "policy" : "ventus_policy_25",
      "lifecycle_date_millis" : 1591903762313,
      "age" : "1.09m",
      "phase" : "hot",
      "phase_time_millis" : 1591903762518,
      "action" : "unfollow",
      "action_time_millis" : 1591903762518,
      "step" : "wait-for-follow-shard-tasks",
      "step_time_millis" : 1591903762547,
      "phase_execution" : {
        "policy" : "ventus_policy_25",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1m"
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1591903762100
      }
    }
  }
}

```

And after waiting more time I see this exception:

[root@elasticmysql ~]# curl -GET [http://elasticmysql:9200/25\_\*/\_ilm/explain?pretty=true](http://elasticmysql:9200/25_*/_ilm/explain?pretty=true)

```auto
{
  "indices" : {
    "25_001" : {
      "index" : "25_001",
      "managed" : true,
      "policy" : "ventus_policy_25",
      "lifecycle_date_millis" : 1591902160104,
      "age" : "23.43m",
      "phase" : "hot",
      "phase_time_millis" : 1591902160292,
      "action" : "rollover",
      "action_time_millis" : 1591902749872,
      "step" : "ERROR",
      "step_time_millis" : 1591903348864,
      "failed_step" : "check-rollover-ready",
      "is_auto_retryable_error" : true,
      "step_info" : {
        "type" : "illegal_argument_exception",
        "reason" : "index name [25_001] does not match pattern '^.*-\\d+$'",
        "stack_trace" : "java.lang.IllegalArgumentException: index name [25_001] does not match pattern '^.*-\\d+$'\n\tat org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.generateRolloverIndexName(TransportRolloverAction.java:241)\n\tat org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:133)\n\tat org.elasticsearch.action.admin.indices.rollover.TransportRolloverAction.masterOperation(TransportRolloverAction.java:73)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.lambda$doStart$3(TransportMasterNodeAction.java:170)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:170)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.start(TransportMasterNodeAction.java:133)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:110)\n\tat org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:59)\n\tat org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:153)\n\tat org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:123)\n\tat org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:151)\n\tat org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:129)\n\tat org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:64)\n\tat org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83)\n\tat org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72)\n\tat org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)\n\tat org.elasticsearch.xpack.core.ClientHelper.executeAsyncWithOrigin(ClientHelper.java:92)\n\tat org.elasticsearch.xpack.core.ClientHelper.executeWithHeadersAsync(ClientHelper.java:155)\n\tat org.elasticsearch.xpack.ilm.LifecyclePolicySecurityClient.doExecute(LifecyclePolicySecurityClient.java:51)\n\tat org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:399)\n\tat org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1234)\n\tat org.elasticsearch.client.support.AbstractClient$IndicesAdmin.rolloverIndex(AbstractClient.java:1736)\n\tat org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:127)\n\tat org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173)\n\tat org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329)\n\tat org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267)\n\tat org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183)\n\tat org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\n"
      },
      "phase_execution" : {
        "policy" : "ventus_policy_25",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "rollover" : {
              "max_age" : "1m"
            }
          }
        },
        "version" : 1,
        "modified_date_in_millis" : 1591902159913
      }
    }
  }
}

```

What I am doing wrong?

Thanks,

Joan.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 11, 2020, 8:10pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777/2 "2020-06-11T20:10:48Z")

</div>

I believe checks are only gone every 5 or 10 minutes, which is ideal when you have more realistic settings. For tests like this you need to change that, but I do not remember how.

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [June 11, 2020, 8:16pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777/3 "2020-06-11T20:16:21Z")

</div>

Hi Joan,

There are two issues, the first is that in order to be used for rollover indices need to end with `-0001` instead of `_0001` (basically a hyphen and then any number).

The second is that the poll interval is 10 minutes by default. For testing, however, you can change this by changing the `indices.lifecycle.poll_interval` setting.

---

<div class="post-metadata">

**Author:** ![jbalaguero](https://avatars.discourse-cdn.com/v4/letter/j/c68b51/32.png) [@jbalaguero](https://discuss.elastic.co/u/jbalaguero)\
**Post date:** [June 11, 2020, 8:42pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777/4 "2020-06-11T20:42:59Z")

</div>

> [@dakrone](#):
>
> indices.lifecycle.poll\_interval

Hi Drakone,

Many thanks.

Do you think that lowering the 'indices.lifecycle.poll\_interval' to 1 min can affect performance?

And a second question: I have an special index named "25-0" that I don't want to include in the ILM. I've been trying to exclude this index using patterns, but I can't. Something like:

```auto
PutIndexTemplateRequest request = new PutIndexTemplateRequest("pattern-25");
request.patterns("25-*","-25_0");

```

But it does not work. If I try `request.patterns("25-*,-25_0");` and exception is thrown.

Any way to do this?

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 9, 2020, 8:43pm UTC](https://discuss.elastic.co/t/indexes-not-being-rotated-removed-using-the-index-lifecycle-management/236777/5 "2020-07-09T20:43:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.