How to change ILM policy?

Elastic Stack Elasticsearch
1

Hi everyone,

I’m using Elasticsearch (8.15.1) for logging in my server environment, and Fleet to distribute agents across the servers.
The indices created are currently using the logs index lifecycle policy, which according to Elasticsearch is now deprecated.

What’s the recommended way to change the ILM policy for indices created through Fleet and Elastic Agent?

From what I’ve seen, there are basically two options:

  1. Create an index template with a higher priority to override the existing ILM setting.

  2. “Fork” the integration package, meaning:

    • Download the Exchange integration package

    • Update its manifest.yml and component templates so that index.lifecycle.name: ilm-logs-exchange

    • Install it as a custom package in Fleet

    • That way, Fleet would handle everything with the new policy.

Are there any other approaches I should consider?

Thanks in advance!

2

Can you upgrade to 8.19.2? This is way more easier to do on later versions where you just need to add the custom ILM policy on the @custom template.

If you cannot upgrade now, you need to follow this specific documentation for 8.15.

You would need to clone the index template for every dataset in every integration that you want to change.

On later versions this step is not required as the default templates already use a @custom component template that will be created on the first time you try to edit it.

3

Hi, and thank you for your reply.

Elasticsearch version is now 8.19.3

Just to clarify, is it enough if I only update the @custom component templates?
For example:
I have an ILM policy called "ilm-logs-firewall", and when I run GET _component_template it shows:

{
  "name": "logs-fortinet_fortigate.log@custom",
  "component_template": {
    "template": {
      "settings": {}
    },
    "_meta": {
      "package": {
        "name": "fortinet_fortigate"
      },
      "managed_by": "fleet",
      "managed": true
    }
  }
}

So it should be enough if I just add:

PUT _component_template/logs-fortinet_fortigate.log@custom
{
  "template": {
    "settings": {
      "index.lifecycle.name": "ilm-logs-firewall"
    }
  }
}

Or have I misunderstood?

Best regards,
Magnus

4

Yes, just adding the ILM setting on the @custom template is enough.

You will need to manually rollover the datastream or wait for it to rollover automatically so new indices can use this policy.