# Need Working ILM example **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [September 6, 2019, 1:13pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427 "2019-09-06T13:13:53Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![P\_Larsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/p_larsen/32/40097_2.png) [@P\_Larsen](https://discuss.elastic.co/u/P_Larsen)\ **Post date:** [September 6, 2019, 1:13pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/1 "2019-09-06T13:13:53Z")
I have tried without success to get ILM working. Can someone post a working index template, rollover alias config, ILM config and anything else I am missing? It seems if you have one of these components incorrect ILM won't work and it won't roll data to warm nodes correctly. We can tweak times later but would like data to move to warm nodes after 1 days, delete after 3 days for testing. We can modify days and index sizes once we get everything working. A working Filebeat config for 7.3 would be perfect. Thanks PL ---
**Author:** ![DanRoscigno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danroscigno/32/70277_2.png) [@DanRoscigno](https://discuss.elastic.co/u/DanRoscigno)\ **Post date:** [September 6, 2019, 4:33pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/2 "2019-09-06T16:33:34Z")
Hi PL, When you deployed Filebeat 7.3 the template, alias, and an ILM policy should have been created, can you tell me: 1. Did the default policy get created? 2. If the policy did get created and you added a warm phase: Can you paste in the output of these two Dev Tool -\> console commands? GET /\_cat/shards/fileb\*?h=index,node GET /\_cat/nodeattrs?v&h=name,attr.name,attr.value Here is my policy: ```auto { "policy": { "phases": { "hot": { "min_age": "0ms", "actions": { "rollover": { "max_age": "1h", "max_size": "50mb", "max_docs": 100 } } }, "warm": { "min_age": "1h", "actions": { "allocate": { "include": {}, "exclude": {}, "require": { "data": "warm" } } } } } } } ``` Regards, Dan ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [September 8, 2019, 1:39am UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/3 "2019-09-08T01:39:07Z")
If you want to try out a fully automated demo, you can pull the code from [https://github.com/xeraa/scale-elasticsearch](https://github.com/xeraa/scale-elasticsearch) and do a `docker-compose up`. Let it generate data for 20min and then you'll have a 3 node cluster (1 hot, 1 warm, 1 cold) with data being moved around automatically. For the demo effect if only keeps data for minutes in each phase, but you should be able to adapt that easily. Also [https://github.com/xeraa/scale-elasticsearch/blob/master/setup/setup.sh](https://github.com/xeraa/scale-elasticsearch/blob/master/setup/setup.sh) shows how this is all set up and held together. ---
**Author:** ![P\_Larsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/p_larsen/32/40097_2.png) [@P\_Larsen](https://discuss.elastic.co/u/P_Larsen)\ **Post date:** [September 9, 2019, 12:28pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/4 "2019-09-09T12:28:43Z")
> [@DanRoscigno](#): > > GET /\_cat/nodeattrs?v&h=name,attr.name,attr.value H is hot nodes W is warm nodes ``` filebeat-7.3.0 SERVERhio02 filebeat-7.3.0 SERVERhio04 filebeat-7.3.0 SERVERwio03 filebeat-7.3.0 SERVERhio05 filebeat-7.3.0 SERVERwio04 filebeat-7.3.0 SERVERwio01 filebeat-7.3.0 SERVERwio05 filebeat-7.3.0 SERVERhio03 filebeat-7.3.0 SERVERwio02 filebeat-7.3.0 SERVERhio01 { "filebeat-7.3.0" : { "version" : 9, "modified_date" : "2019-09-06T18:16:38.891Z", "policy" : { "phases" : { "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "10mb", "max_age" : "1d" } } }, "warm" : { "min_age" : "1d", "actions" : { "allocate" : { "include" : { }, "exclude" : { }, "require" : { "box_type" : "warm" } } } } } } } } ``` ---
**Author:** ![DanRoscigno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danroscigno/32/70277_2.png) [@DanRoscigno](https://discuss.elastic.co/u/DanRoscigno)\ **Post date:** [September 9, 2019, 4:23pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/5 "2019-09-09T16:23:48Z")
Hi PL, I expected to see the values "hot" and "warm" as attributes in the output of `GET /_cat/nodeattrs?v&h=name,attr.name,attr.value`. Did you show that output? ---
**Author:** ![P\_Larsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/p_larsen/32/40097_2.png) [@P\_Larsen](https://discuss.elastic.co/u/P_Larsen)\ **Post date:** [September 9, 2019, 7:27pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/6 "2019-09-09T19:27:49Z")
Here is that output Is there a way to just reload the default filebeat ILM config ? ``` name attr.name attr.value SERVERhio01 ml.machine_memory 67368890368 SERVERhio01 ml.max_open_jobs 20 SERVERhio01 xpack.installed true SERVERhio01 box_type hot SERVERhio04 ml.machine_memory 67368890368 SERVERhio04 ml.max_open_jobs 20 SERVERhio04 xpack.installed true SERVERhio04 box_type hot SERVERmst02 ml.machine_memory 8182042624 SERVERmst02 ml.max_open_jobs 20 SERVERmst02 xpack.installed true SERVERhio05 ml.machine_memory 67368890368 SERVERhio05 ml.max_open_jobs 20 SERVERhio05 xpack.installed true SERVERhio05 box_type hot SERVERhio02 ml.machine_memory 67368890368 SERVERhio02 ml.max_open_jobs 20 SERVERhio02 xpack.installed true SERVERhio02 box_type hot SERVERhio03 ml.machine_memory 67368890368 SERVERhio03 ml.max_open_jobs 20 SERVERhio03 xpack.installed true SERVERhio03 box_type hot SERVERwio02 ml.machine_memory 67368890368 SERVERwio02 ml.max_open_jobs 20 SERVERwio02 xpack.installed true SERVERwio02 box_type warm SERVERmst01 ml.machine_memory 8182050816 SERVERmst01 ml.max_open_jobs 20 SERVERmst01 xpack.installed true SERVERmst03 ml.machine_memory 8182042624 SERVERmst03 ml.max_open_jobs 20 SERVERmst03 xpack.installed true SERVERwio01 ml.machine_memory 67368890368 SERVERwio01 ml.max_open_jobs 20 SERVERwio01 xpack.installed true SERVERwio01 box_type warm SERVERwio03 ml.machine_memory 67368890368 SERVERwio03 ml.max_open_jobs 20 SERVERwio03 xpack.installed true SERVERwio03 box_type warm SERVERwio05 ml.machine_memory 67368890368 SERVERwio05 ml.max_open_jobs 20 SERVERwio05 xpack.installed true SERVERwio05 box_type warm SERVERwio04 ml.machine_memory 67368890368 SERVERwio04 ml.max_open_jobs 20 SERVERwio04 xpack.installed true SERVERwio04 box_type warm SERVERkab01 ml.machine_memory 8182042624 SERVERkab01 ml.max_open_jobs 20 SERVERkab01 xpack.installed true ``` ---
**Author:** ![DanRoscigno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danroscigno/32/70277_2.png) [@DanRoscigno](https://discuss.elastic.co/u/DanRoscigno)\ **Post date:** [September 9, 2019, 7:46pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/7 "2019-09-09T19:46:24Z")
I can ask about the process for removing and reinstalling the policy, but before you do that can you try this please: `GET /_cat/shards/fileb*?h=index,node` ---
**Author:** ![DanRoscigno](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danroscigno/32/70277_2.png) [@DanRoscigno](https://discuss.elastic.co/u/DanRoscigno)\ **Post date:** [September 11, 2019, 2:05pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/8 "2019-09-11T14:05:14Z")
@P_Larsen Let me know how you get along. I would love to see the output of `GET /_cat/shards/fileb*?h=index,node` before you remove and replace the policy. To remove and replace the filebeat ILM policy you have to: - Remove the policy from any existing indices - Delete the policy - Re-run filebeat setup Remove the policy from existing indices: - Go into `Index Management` and change the `Rows per page` at bottom to 100 - type in `filebeat` in the search bar - tick the box above the index list to select all - click on `manage indices` and `Delete Policy` (I forget the exact text, but I think it is at bottom) - repeat for as many pages of indices you have Delete the policy: - From the ILM UI click on `Actions` for the policy and `Delete policy` Run setup: ./filebeat setup --ilm-policy -v ---
**Author:** ![P\_Larsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/p_larsen/32/40097_2.png) [@P\_Larsen](https://discuss.elastic.co/u/P_Larsen)\ **Post date:** [September 11, 2019, 3:31pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/9 "2019-09-11T15:31:15Z")
Thanks for the details I will work on getting filebeat reset again. This appears to be a good working example except the "Creating the first index" step has a typo. ``` Change `"logstash-eagleeye-brofiler"` `: {` to `"logstash-eagleeye-brofilter"` `: {` ``` Other lessons learned: Logstash needs to send data to the index alias. I reduced the settings down to a couple of days so will test using that to see if the data rolls. > **[Optimizing Elasticsearch – Part 2: Index Lifecycle Management](https://blog.nviso.be/2019/06/17/optimizing-elasticsearch-part-2-index-lifecycle-management/)** > > In the previous blog post “Optimize Elasticsearch for log collection – Part 1: reduce the number of shards”, we have seen one solution to recover a cluster suffering from the R… ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 9, 2019, 3:39pm UTC](https://discuss.elastic.co/t/need-working-ilm-example/198427/10 "2019-10-09T15:39:33Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.