You have just one node, you do not need a warm and cold phase as this would do nothing, you can remove those steps.
Also, assuming that your data will rollover after 7 days, it may take 32 days for it to be deleted, you need to check if this time range is enough.
From what you shared it seems that you have a lot of data for a small cluste and may need to reduce both the time that it takes to delete and rollover, and the shard size.
Yes, that seems reasonable. But keep in mind that the following phase and your case the delete phase is calculated from the time of rollover which you have set as a maximum of 7 days in the hot.
So index take 7 days to roll over plus 15 days until delete. You could have data that's 22 days old and then it will delete the 7 days worth of data since the index contain 7 days where the data.
If you're indices, fill up faster. Rollover faster. The same math implies just with a shorter duration
But yes, that's all seems reasonable
No, you do not need to reboot the system.
You may need to force a rollover on the existing indices depending on what version of Elasticsearch and the ILM phase they're in, in order to pick up/ apply the new ILM policy.
What I understand from the above is I can have the data available for last 22 days and once the delete operation occurs I hope the latest 7 days data would be available.
As discussed in the previous post that ILM policy for filebeat was created on 29th Dec-23 as below.
Hot phase (Required)
Rollover: enabled
Maximum primary shard size: 10G
Maximum index size: 10G
Maximum age: 7 days
Now, I am seeing the below message on the kibana GUI which is a production server.
You are editing an existing policy. Any changes you make will affect 32 linked indices(opens in a new tab or window) and **1 linked index template** that are attached to this policy. Alternatively, you can save these changes in a new policy. And 1 linked index template it shows as filebeat-7.17.15.
Question: Is anything wrong here? Please suggest. I am worried if the 1 linked template as mentioned also gets deleted and create a system wide issue.
No that is just a standard awareness warning.
The policy change can/will affect the already linked indices.
The template will not be deleted or changed in any way.
New Indices created by the template will have this new policy applied.
I do want to be clear this is your production system ... if you are concerned you should test in a non-production system.
Yeah, this is on the production server. Also I have implemented the same ILM policy for filbeat in test instance before applying the ILM in production server.
Below is the ss from my test instance where it is not showing anything for 1 linked index template
# Edit policy filebeat
**You are editing an existing policy.** Any changes you make will affect [4 linked indices that are attached to this policy. Alternatively, you can save these changes in a new policy.
Save as new policy
Per our earlier discussions on this thread. The ILM policy was set for filebeat index as below on 29th Dec-23.
Hot phase (Required)
Rollover: enabled
Maximum primary shard size: 10G
Maximum index size: 10G
Maximum age: 7 days
Delete phase
Move data into phase when:15 days old
Note: I was expecting the last 7 days data should have been deleted by now but that's not the case because I could see no changes.
Correct me if I am wrong on my understanding here. Is it going to take 22 days to run the delete operation? when the max age was set to 7 days in hot phase and 15 days to move data into delete phase.
If this is a case then can I change it to lesser value other than 15 days e.g 5 or < 10 days etc in the delete phase instead of waiting for another 15 days more because I can see that the disk space utilization has started piling up.
GET _cat/allocation?v&s=disk.indices&h=shards,disk.indices,disk.used,disk.available,disk.total,disk.percent
I did few changes on my test environment. e.g. below
On the Kibana GUI
Stack Management > Index Lifecycle policies and the filtered out for filebeat
Applied the ILM for filebeat as below
Hot phase (Required)
Rollover: enabled
Maximum primary shard size: 5G
Maximum index size: 5G
Maximum age: 1 days
Delete phase
Move data into phase when:2 days old
Current status when checked using the curl
# curl -X GET "localhost:9200/_cat/indices/filebeat*?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open filebeat-7.17.13-2023.12.29 bgI3JWqJQv-CF3URgOboXA 1 1 360230 0 40.7mb 40.7mb
yellow open filebeat-7.17.13-2023.12.28 aN5saI_kRJmrXBTWhZ7aUg 1 1 157875 0 17.7mb 17.7mb
yellow open filebeat-7.17.13-2023.12.27 3Ff2VmfjT72ElvbbUC3VyA 1 1 267396 0 29.8mb 29.8mb
yellow open filebeat-7.17.13-2024.01.02 uuFIjZdkTYqO6RNEx9B9gg 1 1 4984 0 2.7mb 2.7mb
yellow open filebeat-7.17.13-2024.01.03 MgKear7JQA6hYs5VdPdYxg 1 1 7668 0 2.4mb 2.4mb
yellow open filebeat-7.17.13-2024.01.01 Kj2FCRBUQca5n3mg-yed7Q 1 1 2671 0 1.2mb 1.2mb
yellow open filebeat-7.17.13-2024.01.04 85EFuavPQamUfnLCCu-Ozw 1 1 4507 0 2.6mb 2.6mb
yellow open filebeat-7.17.13-2024.01.05 JbsEFg0tSJCGsJddNgbYgg 1 1 2928 0 1.5mb 1.5mb
yellow open filebeat-7.17.13-2024.01.01-000005 9UnVNkJAS1aaUXEdrt2zZA 1 1 0 0 227b 227b
yellow open filebeat-7.17.13-2023.12.26 MOlB9WyrSOaDDCf-CYJoJQ 1 1 2108355 0 235mb 235mb
yellow open filebeat-7.17.13-2024.01.09-000006 oTXERMP3RFWyUDqy3DMj9A 1 1 0 0 227b 227b
yellow open filebeat-7.17.13-2024.01.09 rIQ_RaOeSIuycyO-Twxizw 1 1 2621 0 1.5mb 1.5mb
Please suggest if there's anything wrong here and why I don't see delete is not occurring.
First Please be patient. There are many questions and your question is no more important than anyone else's. It's only been a couple hours. We will get to your questions if and when we can. There is no guarantee that your question will even get answered. It's only volunteers here.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.