ILM phases rolling out index but keeping

Elastic Stack Elasticsearch
1

Hi All
Below is my ILM policy

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_size": "200gb",
            "max_age": "7d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "7d",
        "actions": {
          "allocate": {
            "require": {
              "data": "warm"
            }
          },
          "readonly": {},
          "set_priority": {
            "priority": 50
          }
        }
      },
      "delete": {
        "min_age": "23d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

This policy rolls over index after 7 days but it is not moving index to warm nodes as rollover reset age of index to 0.

We are expecting to keep 7day old data on hot nodes irrespective size of the index after 7 days data needs to be moved to warm nodes and then ILM should delete the data after 30 days, but right now on hot node data is getting rolled out after 7 days and index age is getting reset to 0 and we have to wait 7 more days to move this data to warm.
Is there any work around for this ? we can keep rolled over indices on hot nodes for 7 days and then we can move them to warm nodes.

2

Welcome to our community! :smiley:

What's the output from GET /_cat/nodeattrs?v?

3

Thanks for using Elasticsearch.

If you'd like the phase transition timings to always (irrespective of rollover) be calculated based on a specific date you can use the origination date as described here

Another option is to set the min_age to 0 for the warm phase. That would translate into "move to the warm phase immediately after rollover". However, this option comes with the caveat that the rollover might be triggered by the "max_size": "200gb" condition in your rollover action before the "max_age" : "7d" is fulfilled if you experience a high (unexpected?) write load. This would mean that the index might advance to the warm (and consequently to the delete) phase earlier than expected. For eg if the warm phase min_age was 0 it could happen that myindex-000001 is rolled over after 5 days (as it reached 200gb), transitions immediately to the warm phase and execute the actions configured there and is deleted after 28 days (as the min_age of the delete phase is 23d + 5d which was the age at which the index was rolled over)

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.