LifeCycle policy doesn't delete index because of wrong index age

Elastic Stack Elasticsearch
1

Hi
I have created an ILM which would delete an index after 5 days of creation. I have disabled the index rollover so my policy has the hot phase and delete phase . Below is the policy for reference .

PUT _ilm/policy/server_activity_lifecycle
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 29
          }
        }
      },
      "delete": {
        "min_age": "5d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

I am observing the issue that some index which are created way around 10-15days earlier are still active(hot phase) and not deleted. I tried using the explain api, the common thing which I could observe with all the index is the age of the index is quite not right.
An example of which is an index named elastic_sink_20220117 which is kind of 14 days old is showing age as "age" : "4.25d".
I didn't understand why the difference ??
I quite get the fact that the age is basically the age after index roll over, but I have disabled the rollover still why the age difference ?
Am I missing something here ?? What can I do to delete the index perfectly after 5 days of creation ?

2

Can you share the output of GET /elastic_sink_20220117/_ilm/explain?human and GET /elastic_sink_20220117/_settings please? Also, when you say that the index is "kind of 14 days old", where are you getting that information?

3

Below are the answer for your question :-

Note :- The above index was deleted by me for some purpose, I am providing the output from similar other index

output of

GET /elastic_sink_20220118/_ilm/explain?human

{
  "indices" : {
    "server-activity-20220118" : {
      "index" : "server-activity-20220118",
      "managed" : true,
      "policy" : "server_activity_lifecycle",
      "lifecycle_date" : "2022-01-28T00:26:32.222Z",
      "lifecycle_date_millis" : 1643329592222,
      "age" : "3.68d",
      "phase" : "hot",
      "phase_time" : "2022-01-28T00:26:32.333Z",
      "phase_time_millis" : 1643329592333,
      "action" : "complete",
      "action_time" : "2022-01-28T00:26:32.458Z",
      "action_time_millis" : 1643329592458,
      "step" : "complete",
      "step_time" : "2022-01-28T00:26:32.458Z",
      "step_time_millis" : 1643329592458,
      "phase_execution" : {
        "policy" : "server_activity_lifecycle",
        "phase_definition" : {
          "min_age" : "0ms",
          "actions" : {
            "set_priority" : {
              "priority" : 100
            }
          }
        },
        "version" : 2,
        "modified_date" : "2022-01-05T08:33:09.388Z",
        "modified_date_in_millis" : 1641371589388
      }
    }
  }
}

GET /elastic_sink_20220118/_settings

{
  "server-activity-20220118" : {
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "server_activity_lifecycle"
        },
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "mapping" : {
          "total_fields" : {
            "limit" : "5000"
          }
        },
        "number_of_shards" : "1",
        "provided_name" : "server-activity-20220118",
        "creation_date" : "1643329592222",
        "priority" : "100",
        "number_of_replicas" : "0",
        "uuid" : "gzZ20Y8ORTCri_ZJ4Z4xfQ",
        "version" : {
          "created" : "7110299"
        }
      }
    }
  }
}

When I say the index is 14 days old I get the info from the part of index name. For each day we have a new index created so for date 18 Jan 2022 the index name will have a suffix of 20220118

4

Okay, even though the date is in the name, it isn't using date math, so the name of the index is incorrect. The "creation_date" : "1643329592222" maps to this index being created on Thu 27 January 2022 at 17:26:32, so it is indeed ~3.7 days old. It was not actually created on the 18th of January.

You can tell if an index was created with date math by checking the "provided_name" setting, which here is "provided_name" : "server-activity-20220118". If it were using date math, then it would have < and > around the date math expression.

2 Likes
5

Ok .. makes sense now. I dug a little based on this info. Looks like the ILM works fine, it does delete the index after the specified time. It seems my connector receives data for previous timestamp and thus a new index is created. I am good for now. Thanks for your quick response.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.