# About the ILM policy implented and on the post observations **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [January 26, 2024, 6:02am UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851 "2024-01-26T06:02:13Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [January 26, 2024, 6:02am UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/1 "2024-01-26T06:02:13Z")
Hello, Recently I have implemented ILM policy on of the production setup. However, the rollover for the existing ILM policy seems to be working fine. But I am seeing indices for other version e.g. 8.x are utilizing the high disk space. ```auto # date && du -h / | sort -rh | head -10 Fri Jan 26 05:42:50 UTC 2024 du: cannot access '/proc/193850/task/193850/fd/4': No such file or directory du: cannot access '/proc/193850/task/193850/fdinfo/4': No such file or directory du: cannot access '/proc/193850/fd/3': No such file or directory du: cannot access '/proc/193850/fdinfo/3': No such file or directory 125G / 119G /var 118G /var/lib 117G /var/lib/elasticsearch/nodes/0/indices 117G /var/lib/elasticsearch/nodes/0 117G /var/lib/elasticsearch/nodes 117G /var/lib/elasticsearch 94G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ/0 94G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ 93G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ/0/index ``` ```auto # date && curl -X GET "localhost:9200/_cat/indices/filebeat*?v" Fri Jan 26 05:51:44 UTC 2024 health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open filebeat-8.11.1 J9GUiOhkThilgzBp0i0RPQ 1 1 200504758 0 95gb 95gb yellow open filebeat-8.11.2 hp8zHe1hTKmfuAJnTsy5fg 1 1 20445253 0 6.4gb 6.4gb yellow open filebeat-8.10.4 HbQfL0pGS_ClJLLBWGFYUQ 1 1 2061898 0 488.6mb 488.6mb yellow open filebeat-7.17.15-2024.01.11 c_aY0AAiQjuJuDmFmCk8Hw 1 1 9924 0 5.6mb 5.6mb yellow open filebeat-8.11.1-2024.01.11 M_CDQqo9QXCjb6VeF0mk4A 1 1 19574081 0 9gb 9gb yellow open filebeat-8.10.4-2024.01.12 7ah5nAK6ShSf8oQhCSar3w 1 1 51352 0 10.2mb 10.2mb yellow open filebeat-8.11.1-2024.01.12 WhYy0DXBRe-85GB1e-ahWw 1 1 13762627 0 6.2gb 6.2gb yellow open filebeat-7.17.15-2024.01.12 S_Xu9N9wQXmIPogZl0a9Aw 1 1 7726 0 4.6mb 4.6mb yellow open filebeat-8.11.2-2024.01.12 ATocymT0SVy1sYqYmivaxg 1 1 1224750 0 405.2mb 405.2mb yellow open filebeat-8.11.2-2024.01.11 4CmGRMeURxGMyatRaPf-Wg 1 1 1763247 0 578.6mb 578.6mb yellow open filebeat-7.17.15-2024.01.22-000066 yUAzRzSCTLaElmzwt3UL5w 1 1 36925 0 12.7mb 12.7mb ``` ```auto # date && curl -X GET "localhost:9200/_cat/indices/*?v" Fri Jan 26 06:00:02 UTC 2024 health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .reporting-2023-12-03 wPLklvtUSMqva6wIzY-DxQ 1 0 1 0 1.9mb 1.9mb yellow open filebeat-8.11.2-2024.01.12 ATocymT0SVy1sYqYmivaxg 1 1 1224750 0 405.2mb 405.2mb green open .apm-agent-configuration tI3nICpPQlWLzlPVFJPg3w 1 0 0 0 227b 227b yellow open filebeat-8.11.2-2024.01.11 4CmGRMeURxGMyatRaPf-Wg 1 1 1763247 0 578.6mb 578.6mb yellow open filebeat-7.17.15-2024.01.22-000066 yUAzRzSCTLaElmzwt3UL5w 1 1 36996 0 12.8mb 12.8mb green open .tasks 7c--yr7bTOe8ADz4vJ6ZsQ 1 0 6 0 41.1kb 41.1kb yellow open filebeat-8.11.1 J9GUiOhkThilgzBp0i0RPQ 1 1 200617915 0 97gb 97gb green open .geoip_databases 4vJGhr5ySUWyUoNXYZxXbw 1 0 43 42 40.4mb 40.4mb yellow open filebeat-8.11.2 hp8zHe1hTKmfuAJnTsy5fg 1 1 20445253 0 6.4gb 6.4gb yellow open filebeat-8.10.4 HbQfL0pGS_ClJLLBWGFYUQ 1 1 2061898 0 488.7mb 488.7mb yellow open filebeat-7.17.15-2024.01.11 c_aY0AAiQjuJuDmFmCk8Hw 1 1 9924 0 5.6mb 5.6mb green open .apm-custom-link rH28BebpTOKIA2KpKxR4aw 1 0 0 0 227b 227b yellow open filebeat-8.10.4-2024.01.12 7ah5nAK6ShSf8oQhCSar3w 1 1 51352 0 10.2mb 10.2mb yellow open filebeat-8.11.1-2024.01.11 M_CDQqo9QXCjb6VeF0mk4A 1 1 19574081 0 9gb 9gb yellow open filebeat-7.17.15-2024.01.12 S_Xu9N9wQXmIPogZl0a9Aw 1 1 7726 0 4.6mb 4.6mb yellow open filebeat-8.11.1-2024.01.12 WhYy0DXBRe-85GB1e-ahWw 1 1 13762627 0 6.2gb 6.2gb green open .kibana_7.17.15_001 XT8tkEdKSASxfkVb5__Xig 1 0 2461 35 3mb 3mb green open .async-search NjkQXv-hS2m8W7GzLjcbFA 1 0 0 0 258b 258b green open .kibana_task_manager_7.17.15_001 XOJAZtH8TxCO_fsH-I0cqA 1 0 17 110959 101.6mb 101.6mb ``` ```auto GET /filebeat-7.17.15-2024.01.22-000066/_ilm/explain { "indices" : { "filebeat-7.17.15-2024.01.22-000066" : { "index" : "filebeat-7.17.15-2024.01.22-000066", "managed" : true, "policy" : "filebeat", "lifecycle_date_millis" : 1705923192297, "age" : "3.75d", "phase" : "hot", "phase_time_millis" : 1705923192530, "action" : "rollover", "action_time_millis" : 1705923192530, "step" : "check-rollover-ready", "step_time_millis" : 1705923192530, "phase_execution" : { "policy" : "filebeat", "phase_definition" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "10gb", "max_primary_shard_size" : "10gb", "max_age" : "7d" } } }, "version" : 13, "modified_date_in_millis" : 1705325399371 } } } } ``` ```auto GET / { "filebeat-7.17.15-2024.01.22-000066" : { "aliases" : { "filebeat-7.17.15" : { "is_write_index" : true } ``` ```auto GET _cat/allocation?v&s=disk.indices&h=shards,disk.indices,disk.used,disk.available,disk.total,disk.percent #! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security. shards disk.indices disk.used disk.total disk.percent 11 25 117.2gb 124gb 242gb 51 ``` Thanks, ---
**Author:** ![yago82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yago82/32/97755_2.png) [@yago82](https://discuss.elastic.co/u/yago82)\ **Post date:** [January 26, 2024, 1:03pm UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/2 "2024-01-26T13:03:39Z")
> [@Ravi\_Pattar](#): > > Hello, > > Recently I have implemented ILM policy on of the production setup. However, the rollover for the existing ILM policy seems to be working fine. But I am seeing indices for other version e.g. 8.x are utilizing the high disk space. > > ```auto > # date && du -h / | sort -rh | head -10 > Fri Jan 26 05:42:50 UTC 2024 > du: cannot access '/proc/193850/task/193850/fd/4': No such file or directory > du: cannot access '/proc/193850/task/193850/fdinfo/4': No such file or directory > du: cannot access '/proc/193850/fd/3': No such file or directory > du: cannot access '/proc/193850/fdinfo/3': No such file or directory > 125G / > 119G /var > 118G /var/lib > 117G /var/lib/elasticsearch/nodes/0/indices > 117G /var/lib/elasticsearch/nodes/0 > 117G /var/lib/elasticsearch/nodes > 117G /var/lib/elasticsearch > 94G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ/0 > 94G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ > 93G /var/lib/elasticsearch/nodes/0/indices/J9GUiOhkThilgzBp0i0RPQ/0/index > > ``` > > ```auto > # date && curl -X GET "localhost:9200/_cat/indices/filebeat*?v" > Fri Jan 26 05:51:44 UTC 2024 > health status index uuid pri rep docs.count docs.deleted store.size pri.store.size > yellow open filebeat-8.11.1 J9GUiOhkThilgzBp0i0RPQ 1 1 200504758 0 95gb 95gb > yellow open filebeat-8.11.2 hp8zHe1hTKmfuAJnTsy5fg 1 1 20445253 0 6.4gb 6.4gb > yellow open filebeat-8.10.4 HbQfL0pGS_ClJLLBWGFYUQ 1 1 2061898 0 488.6mb 488.6mb > yellow open filebeat-7.17.15-2024.01.11 c_aY0AAiQjuJuDmFmCk8Hw 1 1 9924 0 5.6mb 5.6mb > yellow open filebeat-8.11.1-2024.01.11 M_CDQqo9QXCjb6VeF0mk4A 1 1 19574081 0 9gb 9gb > yellow open filebeat-8.10.4-2024.01.12 7ah5nAK6ShSf8oQhCSar3w 1 1 51352 0 10.2mb 10.2mb > yellow open filebeat-8.11.1-2024.01.12 WhYy0DXBRe-85GB1e-ahWw 1 1 13762627 0 6.2gb 6.2gb > yellow open filebeat-7.17.15-2024.01.12 S_Xu9N9wQXmIPogZl0a9Aw 1 1 7726 0 4.6mb 4.6mb > yellow open filebeat-8.11.2-2024.01.12 ATocymT0SVy1sYqYmivaxg 1 1 1224750 0 405.2mb 405.2mb > yellow open filebeat-8.11.2-2024.01.11 4CmGRMeURxGMyatRaPf-Wg 1 1 1763247 0 578.6mb 578.6mb > yellow open filebeat-7.17.15-2024.01.22-000066 yUAzRzSCTLaElmzwt3UL5w 1 1 36925 0 12.7mb 12.7mb > > ``` > > ```auto > # date && curl -X GET "localhost:9200/_cat/indices/*?v" > Fri Jan 26 06:00:02 UTC 2024 > health status index uuid pri rep docs.count docs.deleted store.size pri.store.size > green open .reporting-2023-12-03 wPLklvtUSMqva6wIzY-DxQ 1 0 1 0 1.9mb 1.9mb > yellow open filebeat-8.11.2-2024.01.12 ATocymT0SVy1sYqYmivaxg 1 1 1224750 0 405.2mb 405.2mb > green open .apm-agent-configuration tI3nICpPQlWLzlPVFJPg3w 1 0 0 0 227b 227b > yellow open filebeat-8.11.2-2024.01.11 4CmGRMeURxGMyatRaPf-Wg 1 1 1763247 0 578.6mb 578.6mb > yellow open filebeat-7.17.15-2024.01.22-000066 yUAzRzSCTLaElmzwt3UL5w 1 1 36996 0 12.8mb 12.8mb > green open .tasks 7c--yr7bTOe8ADz4vJ6ZsQ 1 0 6 0 41.1kb 41.1kb > yellow open filebeat-8.11.1 J9GUiOhkThilgzBp0i0RPQ 1 1 200617915 0 97gb 97gb > green open .geoip_databases 4vJGhr5ySUWyUoNXYZxXbw 1 0 43 42 40.4mb 40.4mb > yellow open filebeat-8.11.2 hp8zHe1hTKmfuAJnTsy5fg 1 1 20445253 0 6.4gb 6.4gb > yellow open filebeat-8.10.4 HbQfL0pGS_ClJLLBWGFYUQ 1 1 2061898 0 488.7mb 488.7mb > yellow open filebeat-7.17.15-2024.01.11 c_aY0AAiQjuJuDmFmCk8Hw 1 1 9924 0 5.6mb 5.6mb > green open .apm-custom-link rH28BebpTOKIA2KpKxR4aw 1 0 0 0 227b 227b > yellow open filebeat-8.10.4-2024.01.12 7ah5nAK6ShSf8oQhCSar3w 1 1 51352 0 10.2mb 10.2mb > yellow open filebeat-8.11.1-2024.01.11 M_CDQqo9QXCjb6VeF0mk4A 1 1 19574081 0 9gb 9gb > yellow open filebeat-7.17.15-2024.01.12 S_Xu9N9wQXmIPogZl0a9Aw 1 1 7726 0 4.6mb 4.6mb > yellow open filebeat-8.11.1-2024.01.12 WhYy0DXBRe-85GB1e-ahWw 1 1 13762627 0 6.2gb 6.2gb > green open .kibana_7.17.15_001 XT8tkEdKSASxfkVb5__Xig 1 0 2461 35 3mb 3mb > green open .async-search NjkQXv-hS2m8W7GzLjcbFA 1 0 0 0 258b 258b > green open .kibana_task_manager_7.17.15_001 XOJAZtH8TxCO_fsH-I0cqA 1 0 17 110959 101.6mb 101.6mb > > ``` > > ```auto > GET /filebeat-7.17.15-2024.01.22-000066/_ilm/explain > > { > "indices" : { > "filebeat-7.17.15-2024.01.22-000066" : { > "index" : "filebeat-7.17.15-2024.01.22-000066", > "managed" : true, > "policy" : "filebeat", > "lifecycle_date_millis" : 1705923192297, > "age" : "3.75d", > "phase" : "hot", > "phase_time_millis" : 1705923192530, > "action" : "rollover", > "action_time_millis" : 1705923192530, > "step" : "check-rollover-ready", > "step_time_millis" : 1705923192530, > "phase_execution" : { > "policy" : "filebeat", > "phase_definition" : { > "min_age" : "0ms", > "actions" : { > "rollover" : { > "max_size" : "10gb", > "max_primary_shard_size" : "10gb", > "max_age" : "7d" > } > } > }, > "version" : 13, > "modified_date_in_millis" : 1705325399371 > } > } > } > } > > ``` > > ```auto > GET / > > { > "filebeat-7.17.15-2024.01.22-000066" : { > "aliases" : { > "filebeat-7.17.15" : { > "is_write_index" : true > } > > ``` > > ```auto > GET _cat/allocation?v&s=disk.indices&h=shards,disk.indices,disk.used,disk.available,disk.total,disk.percent > > #! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security. > shards disk.indices disk.used disk.total disk.percent > 11 > 25 117.2gb 124gb 242gb 51 > > ``` > > Thanks, Hi, This index is not managed by the ILM policy you have in place, which is why it's not being rolled over or deleted. If you want to apply your ILM policy to this index, you can do so by updating the index settings. ```auto PUT /filebeat-8.11.1/_settings { "index.lifecycle.name": "filebeat" } ``` Regards ---
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [January 27, 2024, 5:08am UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/3 "2024-01-27T05:08:04Z")
> [@yago82](#): > > Hi, > > This index is not managed by the ILM policy you have in place, which is why it's not being rolled over or deleted. > > If you want to apply your ILM policy to this index, you can do so by updating the index settings. > > ```auto > PUT /filebeat-8.11.1/_settings > { > "index.lifecycle.name": "filebeat" > } > > ``` > > Regards Hello, Thank you for the response. Will test and update here on the same thread. Thanks, ---
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [January 30, 2024, 5:55am UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/4 "2024-01-30T05:55:35Z")
> [@yago82](#): > > This index is not managed by the ILM policy you have in place, which is why it's not being rolled over or deleted. > > If you want to apply your ILM policy to this index, you can do so by updating the index settings. > > ```auto > PUT /filebeat-8.11.1/_settings > { > "index.lifecycle.name": "filebeat" > } > > ``` Hello, Please find the GET request details below for the filebeat-8.11.1 ```auto { "filebeat-8.11.1" : { "settings" : { "index" : { "routing" : { "allocation" : { "include" : { "_tier_preference" : "data_content" } } }, "number_of_shards" : "1", "provided_name" : "filebeat-8.11.1", "creation_date" : "1705083736588", "number_of_replicas" : "1", "uuid" : "J9GUiOhkThilgzBp0i0RPQ", "version" : { "created" : "7171599" } } } } } ``` Question: simply running the PUT request as mentioned above will fix the issue? ```auto PUT /filebeat-8.11.1/_settings { "index.lifecycle.name": "filebeat" } ``` Because when ILM policy was implemented for filebeat-7.x before it was the below procedure which was followed since the logic was filebeat --\> Logstash --\> Elasticsearch 1. Fix Logstash output 2. Point filebeat output to elasticsearch 3. ./filebeat setup -e (your command is not correct) 4. Point filebeat output back to logstash 5. Start Filebeat The same procedure has to be run when applying the ILM for filebeat-8.11.1? Please suggest. Also, as a note which I have informed in the previous forum discussions as below. And 8.x Elastic Handles all this slightly different with Datastreams and if you are mizxing 7.x and 8.x Beats vs Logstash vs Elasticsearch you may have issues... If you use 8.x Elasticsearch and Logstash you will need to use the output config shown on this page.... > **[Data streams | Elasticsearch Guide \[8.12\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html)** Thanks, Ravi ---
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [January 31, 2024, 1:46pm UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/6 "2024-01-31T13:46:52Z")
> [@yago82](#): > > If you want to apply your ILM policy to this index, you can do so by updating the index settings. > > ```auto > PUT /filebeat-8.11.1/_settings > { > "index.lifecycle.name": "filebeat" > } > > ``` Hello, On my recent post along with the details and information shared. will running the below step alone will fix the issue? Please suggest ```auto PUT /filebeat-8.11.1/_settings { "index.lifecycle.name": "filebeat" } ``` Thanks, ---
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [February 1, 2024, 4:41pm UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/7 "2024-02-01T16:41:21Z")
Hello, Since this issue I am facing on the production, please can you confirm on my recent posts. Thanks, Ravi ---
**Author:** ![Ravi\_Pattar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ravi_pattar/32/124209_2.png) [@Ravi\_Pattar](https://discuss.elastic.co/u/Ravi_Pattar)\ **Post date:** [February 5, 2024, 1:16pm UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/8 "2024-02-05T13:16:52Z")
> [@yago82](#): > > If you want to apply your ILM policy to this index, you can do so by updating the index settings. > > ```auto > PUT /filebeat-8.11.1/_settings > { > "index.lifecycle.name": "filebeat" > } > > ``` Hi @yago82 Can you please confirm if this in not going to mess up with the existing ILM policy running on the live server? Because the ILM policy on the live server exists for filebeat version 7.17.15 and is in operational mode. Please find the details below. GET /\_ilm/policy/filebeat ```auto { "filebeat" : { "version" : 13, "modified_date" : "2024-01-15T13:29:59.371Z", "policy" : { "phases" : { "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "10gb", "max_primary_shard_size" : "10gb", "max_age" : "7d" } } }, "delete" : { "min_age" : "3d", "actions" : { "delete" : { "delete_searchable_snapshot" : true } } } } }, "in_use_by" : { "indices" : [ "filebeat-7.17.15-2024.01.12", "filebeat-7.17.15-2024.02.05-000068", "filebeat-7.17.15-2024.01.29-000067" ], "data_streams" : [], "composable_templates" : [ "filebeat-7.17.15" ] } } } ``` GET \_ilm/status ```auto { "operation_mode" : "RUNNING" } ``` Thanks, ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 4, 2024, 1:17pm UTC](https://discuss.elastic.co/t/about-the-ilm-policy-implented-and-on-the-post-observations/351851/9 "2024-03-04T13:17:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.