Managing ILM policies

[AlessandroKP (Alessandro)]

Hi all,
I would like some information about index lifecycle management.
I have some data stored in ES under indices whose names are of the form:
my-logs-A.2019-01, my-logs-A.2019-02, my-logs-A.2019-03, ...
my-logs-B.2019-01, my-logs-B.2019-02, my-logs-B.2019-03, ... and so on.
Each one refers to data collected in a specific month.
In this example I have two 'types' of data: my-logs-A and my-logs-B.
The data enters ES through a Logstash pipeline that decides the index where to inject the documents and the template.

I know that I can manage the lifecycle of a given index by applying a certain ILM policy to the corresponding index template.
For example, I have defined this index template:

    {
      "my-logs-A-template" : {
        "index_patterns" : ["my-logs-A.*],
        "settings" : {
          "index" : {
            "lifecycle" : {
              "name" : "retention_3_months"
            },
            "number_of_shards" : "1",
            "number_of_replicas" : "0"
          }
        }
      }
    }

and the policy called "retention_3_months"

{
  "retention_3_months" : {
    "policy" : {
      "phases" : {
        "delete" : {
          "min_age" : "90d",
          "actions" : {
            "delete" : { }
          }
        }
      }
    }
  }
}

This ILM policy applies to all indices of the form my-logs-A.* and that's good.
Now, I can create an other template for the data stored in my-logs-B.* and assign a different ILM policy to it.

What I would like to do is to define a single template for both types of data, that is:

{
  "my-logs-template" : {
    "index_patterns" : [
      "my-logs-*"
    ],
    "settings" : {
		...
    }
  }
}

but decide which ILM policy to apply to specific index patterns based on some expression present in the index name.
In my example, if the index name matches the pattern "my-logs-A.*", then I need to apply policy retention_3_months;
if it matches "my-logs-B.*" then I need to apply policy retention_6_months.
But the two index pattern must have the same template.

Is there a way to select which ILM to apply based on a condition, for indices having the same template?

[gbrown (Gordon Brown)]

This isn't currently possible, unfortunately - you'll have to use multiple templates.

However, if multiple templates match, they are merged - so you could have one template with most of the settings and the mapping with the pattern my-logs-*, and two other templates that just have the lifecycle policy name that match my-logs-A* and my-logs-B*`. That would let you have the shared settings/mapping data in one template, and add onto or override that as needed.