Index Lifecycle not being applied

Elastic Stack Elasticsearch
1

I'm having problems wth index lifecycle policy.

As you can see in the print "ilm-index-traces-apm", there are 2 indices that are assigned to the lifecycle policy "dexco-lifecycle-7days".

The "dexco-lifecycle-7days" policy can be viewed below:

{
  "dexco-lifecycle-7days": {
    "version": 3,
    "modified_date": "2022-10-20T14:03:55.766Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "20gb",
              "max_age": "6d"
            },
            "set_priority": {
              "priority": 100
            }
          }
        },
        "delete": {
          "min_age": "6d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    }
}

Could someone help me identify why the
ilm-index-traces-apm
oldest index ".ds-traces-apm-default-2022.10.19-000167" created on 10/19/2022 (today is the 27th) was not deleted? In the policy, it was defined to exclude indices aged 6 days and this is not being respected.

Can you help?

2

Hi @OscarFilho

How are you sure those indices have the ILM policy applied?

They may be but nothing you showed shows that.

You can go the ILM policies and see what indices are under management.. you can also see what state they are in...with respect to ILM.

Also to be clear what the policy states is ..

Hot
Rollover at 20GB or 6 Days whichever comes first.

Then you delete says 6 days after rollover...

So if it took 6 days to rollover which it looks like it did.... Then delete would not happen for another 6 days after that so up to 12 days total.

All subsequent actions are relative to the rollover time, not creation of the index.

Not sure what you mean by this exclude

3

Just curious, is this how ILM works when you use rollover? Because for normal index without rollover ILM does not work this way, each phase is based in the creation date of the index, so a delete phase after 6 days will delete the index 6 days after it was created.

There is anything in the documentation about this difference?

4

The min_age value is relative to the rollover time, not the index creation time.

1 Like
5

Funny. I've never used ILM without rollover so that's kind of burned into my brain. I didn't really think about using ILM without rollover.

6

In my case I use Logstash to create daily indices, the ILM is used to move indices from hot nodes to warm nodes and delete after a couple of time, and in this case it uses the creation time.

2 Likes
7

Thanks for the reply, stephenb and leandrojmp!

I meant DELETE instead EXCLUDE.

I attached (file datastreams-created.png) the list of datastreams that were created from a policy I changed:

datastreams-created
datastreams-created1655×462 56.6 KB

I also attached (file datastreams-policy-applied.png) the ILM policy that was applied to all datastreams created:

datastreams-policy-applied
datastreams-policy-applied1661×664 93.8 KB

Below is the definition of the "dexco-graylog-logs-policy" policy:

{
  "dexco-graylog-logs-policy": {
    "version": 6,
    "modified_date": "2022-10-28T13:47:44.742Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_size": "5gb",
              "max_primary_shard_size": "5gb",
              "max_age": "1d"
            },
            "set_priority": {
              "priority": 101
            }
          }
        },
        "delete": {
          "min_age": "5d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [
        ".ds-graylog-elk-2022.10.31-000011",
        ".ds-graylog-elk-2022.10.28-000006",
        ".ds-graylog-elk-2022.10.28-000005",
        ".ds-graylog-elk-2022.10.29-000007",
        ".ds-graylog-elk-2022.10.27-000003",
        ".ds-graylog-elk-2022.10.30-000008",
        ".ds-graylog-elk-2022.10.31-000009"
      ],
      "data_streams": [
        "graylog-elk"
      ],
      "composable_templates": [
        "greylogtemplate",
        "graylog-elk"
      ]
    }
  }
}

I would like to have an index rotation based on the age of 1 day maximum for each index, and after 5 days after each rollover made on top of each index, there would be a deletion of the oldest index.
Could you guide me how to do this? I couldn't quite understand how ILM works.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.