Hi! Please help me. I have configured ILM to rollover the index when it reaches 120GB or 7 days. I have a problem that ILM does not rollover the index because it thinks that the index size is for example 117GB, when in fact it is already about 130GB. Why can this happen?
Hi @kruzadmn,
Welcome to the community! Are there any errors in your logs at all?
There are some useful details on troubleshooting common issues, and also diagnosing non-deleting in this blog and the documentation that might help you figure out what's going on. Have you tried any of those?
Thank you @carly.richmond . Unfortunately, I didn't see any errors in the logs. I also looked at the documentation you sent but did not find an answer to my issue. But by the way, now the index is rolover correctly. Although the new index that was created again does not match the real size. this is a bit strange.
Ok, there is a common issue in the blog mentioned above that a delete phase needs to be configured. Have you configured a delete phase? Can you share your policy configuration?
Yes, I have configured the deletion. But I decided to check the other indexes and found that the other indexes have the same problem. Some indexes have a difference of a few GB, while others can be up to 10GB. It seems to me that this is a problem with the cluster settings. I also noticed that when a new index is created, it takes a long time to initialize.
@carly.richmond Hi again! Do you have any ideas on how to solve this problem? Thanks!
Hi @kruzadmn,
It's difficult to advise what the issue is without being able to see the ILM policy or having any knowledge of the cluster settings that you think may be contributing to the problem.
Can you share your policy configuration and the output of _ilm/explain API?
Policy configuration.
{
"id": "logs-windows.system-servers",
"seqNo": 1094411,
"primaryTerm": 13,
"policy": {
"policy_id": "logs-windows.system-servers",
"description": "All Windows servers system logs",
"last_updated_time": 1689755591201,
"schema_version": 18,
"error_notification": null,
"default_state": "hot",
"states": [
{
"name": "hot",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"rollover": {
"min_size": "120gb",
"min_index_age": "24h"
}
}
],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_rollover_age": "1d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"retry": {
"count": 3,
"backoff": "exponential",
"delay": "1m"
},
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"logs-windows.system-servers"
],
"priority": 1,
"last_updated_time": 1685608155190
}
]
}
}
Explain
{
".ds-logs-windows.system-servers-000029": {
"index.plugins.index_state_management.policy_id": "logs-windows.system-servers",
"index.opendistro.index_state_management.policy_id": "logs-windows.system-servers",
"index": ".ds-logs-windows.system-servers-000029",
"index_uuid": "RIqPp7svQICg4oU5TfjmnQ",
"policy_id": "logs-windows.system-servers",
"policy_seq_no": -2,
"policy_primary_term": 0,
"rolled_over": false,
"index_creation_date": 1689837466367,
"state": {
"name": "hot",
"start_time": 1689837795943
},
"action": {
"name": "rollover",
"start_time": 1689838152973,
"index": 0,
"failed": false,
"consumed_retries": 0,
"last_retry_time": 0
},
"step": {
"name": "attempt_rollover",
"start_time": 1689838152973,
"step_status": "condition_not_met"
},
"retry_info": {
"failed": false,
"consumed_retries": 0
},
"info": {
"message": "Pending rollover of index [index=.ds-logs-windows.system-servers-000029]",
"conditions": {
"min_index_age": {
"condition": "1d",
"current": "1.5h",
"creationDate": 1689837466367
},
"min_size": {
"condition": "120gb",
"current": "13.3gb"
}
}
},
"enabled": true
},
"total_managed_indices": 1
}
OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.
(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns 
)
Hi @kruzadmn,
It looks like you're using OpenSearch rather than Elasticsearch. Which version are you using? Have you raised your question in the OpenSearch forum?
Yes, I use Opensearch because unfortunately my company can't afford to buy the full version of Elastic, and I'm interested in some of the features that are available in the full version Elasticsearch
I am using the latest version of Opensearch 2.8.0. Yes, I have asked this question on the forum, in Slack workspace, OpsterGPT and unfortunately I did not get an answer.
Sorry to hear that. Sadly there are some key differences in how Elasticsearch ILM and OpenSearch ISM works so I would recommend going back to the OpenSearch forum.
Best of luck solving your issue!
Anyway, thanks for trying to help.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.