Problem with Index lifecycle management

Elastic Stack Elasticsearch
1

Hi All...
Here I am trying to create an Index LifeCycle management policy which

  1. Initially store the index in the hot node for maximum 10 minutes.
  2. Next, that index should move to warm node, forcemerge and Shrink operations need to be performed if the minimum age of the index is 10 minutes older.
  3. The index should move to the cold node if the minimum age of the index is 15 minutes.
  4. The index should be deleted if the min_age of the index is 20 minutes.
    And Here the ilm policy
   PUT _ilm/policy/hot-warm-cold-delete-20min
    {
      "policy": {
        "phases": {
          "hot": {
            "actions": {
              "rollover": {
                "max_age":"10m"
              },
              "set_priority": {
                "priority": 50
              }
            }
          },
          "warm": {
            "min_age": "10m",
            "actions": {
              "forcemerge": {
                "max_num_segments": 1
              },
              "shrink": {
                "number_of_shards": 1
              },
              "allocate": {
                "require": {
                  "box_type": "warm"
                }
              },
              "set_priority": {
                "priority": 25
              }
            }
          },
          "cold": {
            "min_age": "15m",
            "actions": {
              "set_priority": {
                "priority": 0
              },
              "freeze": {},
              "allocate": {
                "require": {
                  "box_type": "cold"
                }
              }
            }
          },
          "delete": {
            "min_age": "20m",
            "actions": {
              "delete": {}
            }
          }
        }
      }
    }

And Here is the template that I used

 PUT _template/hot-warm-cold-delete-60days-template
 {
 "order": 10,
 "index_patterns": ["titanic-*"],
"settings": {
 "number_of_shards": 2,
 "number_of_replicas": "0",
"index.routing.allocation.require.box_type": "hot",
"index.lifecycle.name": "hot-warm-cold-delete-20min"
 }
 }

Then I have indexed titanic-1 index and two shards were allowcated on hot node, then I waited for 10 minutes and more but shardes were not relocated to warm, cold nodes and index is also not deleted.
And I used logstash to index the data and enabled ilm_enabled => true in output elasticsearch plugin.
And I have 3 node cluster and each node has one of the following attribute
node.attr.box_type: hot
node.attr.box_type: warm
node.attr.box_type: cold

Please correct me if I am doing wrong

Thank You...

2

I belueve ILM by default only checks every 10 minutes which is fine for real use cases but not optimal for these kind of tests.

3

Thank you for your response.
So If I decreasing ILM checks time to 5 minutes, will my policy workes?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.