We keep seeing the issue described here where Elasticsearch is not rolling over indices properly.
[2021-09-13T12:39:16,671][ERROR][o.e.x.i.IndexLifecycleRunner] [ecom-repository01] policy [test-policy] for index [errors-test-000021] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [errors-test] does not point to index [errors-test-000021]
at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:104) [x-pack-core-7.7.0.jar:7.7.0]
at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173) [x-pack-ilm-7.7.0.jar:7.7.0]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329) [x-pack-ilm-7.7.0.jar:7.7.0]
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267) [x-pack-ilm-7.7.0.jar:7.7.0]
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183) [x-pack-core-7.7.0.jar:7.7.0]
- How can we resolve this permanently? My feeling is ILM in general is a bit flaky.
- Is there a way to query such orphaned indices so we can re-attach them?