Hot-warm policy

Elastic Stack Elasticsearch
1

Hello,

I would like to create a Hot-warm policy , and the index should rollover when the index is 20Gb of size or max_age equal to 30days BUT, if the size condition occur before the age condition, the index should rollover but the data have to stay in the hot node till the max_age condition occur. and then the data should be in warm data for 5 months and then deleted.

Example : if after 15days the index is 20gb, the index rollover but don't leave the hot data node till it's age is 30 days, so should stay other 15 days in the hot data before if goes into the warm data (Hope I explain it well :sweat_smile:)

SO I created this policy

PUT _ilm/policy/hot-warm-cold-delete-6months-policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size":"20gb",
            "max_age":"30d"
          },
          "set_priority": {
            "priority": 50
          }
        }
      },
      "warm": {
        "min_age": "30d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          },
          "shrink": {
            "number_of_shards": 1
          },
          "allocate": {
            "require": {
              "data": "warm"
            }
          },
          "set_priority": {
            "priority": 25
          }
        }
      },
      "delete": {
        "min_age": "150d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

but if I understand well, this means that the index will be sent to the warm data after 30 days of the rollover and it doesn't work as I want exactly

Could you tell me please if what I am trying to do is possible with ILM ?

Thanks for your help :slight_smile:

2

I haven't used it, but I think this is "lifecycle.parse_origination_date"

1 Like
3

It is, as mentioned by Len, you can do this with the parse_origination_date setting, check out this blog for an example:

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.