# Some problems about index life cycle **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [May 3, 2021, 9:22am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953 "2021-05-03T09:22:19Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ktpktr0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ktpktr0/32/87157_2.png) [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Post date:** [May 3, 2021, 9:22am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/1 "2021-05-03T09:22:19Z")
I use logstash to add fields to dynamically generate indexes. ```auto index => "%{[service]}-%{+YYYY.MM.dd}" ``` I configure the following index template to match the generated index ```auto { "index_templates" : [ { "name" : "pre-docker", "index_template" : { "index_patterns" : [ "docker_console_pre*" ], "composed_of" : [ "games-mappings", "pre-games-settings" ], "priority" : 112, "version" : 1 } } ] } ``` ```auto { "component_templates" : [ { "name" : "pre-games-settings", "component_template" : { "template" : { "settings" : { "index" : { "lifecycle" : { "name" : "pre", "rollover_alias" : "docker_console_pre" }, "routing" : { "allocation" : { "require" : { "box_type" : "hot" } } }, "number_of_shards" : "3", "number_of_replicas" : "1" } } }, "version" : 1 } } ] } ``` When I was testing the rollover update in the index life cycle, I encountered several problems: 1. Index template cannot write alias directly. It must append alias to index after index creation. Otherwise, the following error will be reported: ```auto illegal_ argument_ exception: Rollover alias [docker_ console_ pre] can point to multiple indices, found duplicated alias [docker_ console_ pre] in index template [pre-docker] ``` 1. When the index starts to scroll, the 000001 index will be automatically deleted, but logstash will generate a new 00001 index and start writing data. It will report the error mentioned earlier soon. 2. Since the index is generated once a day, the errors mentioned above also occur. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 4, 2021, 12:06am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/2 "2021-05-04T00:06:42Z")
You will need to drop the `%{+YYYY.MM.dd}` from your output section, so that you are writing to the ILM alias name to start with. What does your ILM policy look like? ---
**Author:** ![ktpktr0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ktpktr0/32/87157_2.png) [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Post date:** [May 4, 2021, 1:09am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/3 "2021-05-04T01:09:38Z")
If you delete %{+YYYY.MM.dd}, the roll alias will be the same as the index, and the life cycle strategy is as follows: ```auto { "pre" : { "version" : 5, "modified_date" : "2021-05-03T15:20:13.108Z", "policy" : { "phases" : { "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "25gb", "max_age" : "60s" }, "set_priority" : { "priority" : 100 } } }, "delete" : { "min_age" : "120s", "actions" : { "delete" : { "delete_searchable_snapshot" : true } } }, "warm" : { "min_age" : "0ms", "actions" : { "allocate" : { "include" : { }, "exclude" : { }, "require" : { "box_type" : "warm" } }, "set_priority" : { "priority" : 50 } } } } } } } ``` ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 4, 2021, 1:16am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/4 "2021-05-04T01:16:38Z")
> [@ktpktr0](#): > > `"max_age" : "60s"` That's 60 **seconds** , I don't think that is what you want. You may want to check the rest of your `min_age` settings as they all seem wrong. > [@ktpktr0](#): > > If you delete %{+YYYY.MM.dd}, the roll alias will be the same as the index No, because ILM defaults to adding numbers to the end of the provided alias. You need to be indexing data into the write alias that you defined. ---
**Author:** ![ktpktr0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ktpktr0/32/87157_2.png) [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Post date:** [May 4, 2021, 1:26am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/5 "2021-05-04T01:26:46Z")
The life cycle is short and is used for testing. Are the following steps correct: 1. Configure lifecycle policy 2. Configure index template 3. Set the index of logstash to index =\> "%{[service]} 1. Delete the old index for logstash rebuild. 2. See if the lifecycle works ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 4, 2021, 1:30am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/6 "2021-05-04T01:30:30Z")
If you've followed the documentation here - [Configure a lifecycle policy | Elasticsearch Guide [7.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.12/set-up-lifecycle-policy.html) - then you should be on the right path. What you have outlined looks right from a high level. > [@ktpktr0](#): > > The life cycle is short and is used for testing. Ok, but the reality is that it's not going to work with such short timeframes due to the way it's polled. ---
**Author:** ![ktpktr0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ktpktr0/32/87157_2.png) [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Post date:** [May 4, 2021, 2:52am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/7 "2021-05-04T02:52:19Z")
When I follow this procedure, logstash generates a docker\_ console\_ Index of pre. When scrolling updates, it prompts the following error: llegal\_ argument\_ exception: index.lifecycle.rollover\_ alias [docker\_ console\_ pre] does not point to index [docker\_ console\_ pre] ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 4, 2021, 2:54am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/8 "2021-05-04T02:54:26Z")
It'd be helpful if you could post a full reproduction of what you are doing. ---
**Author:** ![ktpktr0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ktpktr0/32/87157_2.png) [@ktpktr0](https://discuss.elastic.co/u/ktpktr0)\ **Post date:** [May 4, 2021, 3:20am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/9 "2021-05-04T03:20:43Z")
Please refer to the life cycle policy and index template listed above. Logstash configuration is as follows: ```auto output { elasticsearch { hosts => ["192.168.1.165:9200"] ssl => true ssl_certificate_verification => true cacert => "/etc/logstash/ssl/logstash.pem" index => "%{[service]}" } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 1, 2021, 3:21am UTC](https://discuss.elastic.co/t/some-problems-about-index-life-cycle/271953/10 "2021-06-01T03:21:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.