# Setting different ilm policy based on index name

**URL:** <https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 27, 2020, 9:15am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497 "2020-05-27T09:15:01Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 9:15am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/1 "2020-05-27T09:15:01Z")

</div>

I use "indices" syntax to create a different indexes based on a module name.  
I want to apply different ilm policy based on index name?  
How can I achieve that?

```
  indices:
    - index: "filebeat-netflow-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "netflow"

    - index: "filebeat-cisco-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "cisco"

```

I am currently on version 7.7.0

---

<div class="post-metadata">

**Author:** ![KoettingSimon](https://avatars.discourse-cdn.com/v4/letter/k/ba9def/32.png) [@KoettingSimon](https://discuss.elastic.co/u/KoettingSimon)\
**Post date:** [May 27, 2020, 10:10am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/2 "2020-05-27T10:10:37Z")

</div>

You can define 2 index templates in elasticsearch.  
In the index template you can define the ilm policy which should be used, see [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html#ilm-gs-apply-policy)

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 11:51am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/3 "2020-05-27T11:51:37Z")

</div>

Thank you for your repley. After reading It I connected the dots and did the following.

1.) disabled ilm in filebeat

```
#setup.ilm.enabled: auto
#setup.ilm.rollover_alias: "filebeat"
#setup.ilm.pattern: "%{now/d}-rolled"
#setup.ilm.check_exists: true
#setup.ilm.policy_file: /etc/filebeat/ilm_policy_cisco.txt
#setup.ilm.overwrite: true

```

2.) copied the filebeat template and change the  
index pattern and ILM policy accordingly

```
"index": {
    "lifecycle": {
      "name": "filebeat-netflow",
      "rollover_alias": "filebeat-netflow

```

3.) Created new policy and use "action" button to assign it to policy (maybe this was already done but I wanted to be sure)

After deleting indices I don't see the copy of idecies that are already rolled and throwing error in the dashboards.

I hope it will work now. I will see tomorrow.

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 12:36pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/4 "2020-05-27T12:36:33Z")

</div>

I still get

```
"type" : "illegal_argument_exception",
        "reason" : "index.lifecycle.rollover_alias [filebeat-cisco] does not point to index [filebeat-cisco-2020.05.27]"

```

but i think adding -000001 to index will resolve the issue

> [@Index lifecycle error - illegal\_argument\_exception: index.lifecycle.rollover\_alias](https://discuss.elastic.co/t/index-lifecycle-error-illegal-argument-exception-index-lifecycle-rollover-alias/207900):
>
> Hello Team, Could you please help me with sorting out the following error while implementing ILM? Referred link [https://www.elastic.co/guide/en/elasticsearch/reference/7.4/getting-started-index-lifecycle-management.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/getting-started-index-lifecycle-management.html) Index lifecycle error illegal\_argument\_exception: index.lifecycle.rollover\_alias [temprule] does not point to index [templogs-2019.11] Current action rollover Failed step check-rollover-ready Current phase hot Show phase definition { "index.blocks.read\_only\_allow\_de…

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 1:16pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/5 "2020-05-27T13:16:30Z")

</div>

Nooo it still does thrown the same error "illegal\_argument\_exception"

```
  indices:
    - index: "filebeat-netflow-%{+yyyy.MM.dd}-000001"
      when.equals:
        event.module: "netflow"

    - index: "filebeat-cisco-%{+yyyy.MM.dd}-000001"
      when.equals:
        event.module: "cisco"

```

I think i missing [step 3 bootstraping](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html). Can I do It from kibana interface?

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 1:26pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/6 "2020-05-27T13:26:38Z")

</div>

After I had added aliases to the temples error disappeared but not for long 😕

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4ce5d4bec792b0e6b5ebe8429265c9a24c4fb853.png)

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 2:20pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/7 "2020-05-27T14:20:38Z")

</div>

My elasticsearch logs says

`java.lang.IllegalArgumentException: source alias [filebeat-cisco] does not point to a write index`

At this point, I am sure that this error occur because there is no

`"is_write_index": true`

in my template

but how can I add this? Doing that through edit template step 4 via kibana is not working.

---

<div class="post-metadata">

**Author:** ![KoettingSimon](https://avatars.discourse-cdn.com/v4/letter/k/ba9def/32.png) [@KoettingSimon](https://discuss.elastic.co/u/KoettingSimon)\
**Post date:** [May 27, 2020, 3:12pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/9 "2020-05-27T15:12:56Z")

</div>

You do not need to specify a alias in the template.  
All you need to define in the template is described [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html#ilm-gs-apply-policy)

So just delete the alias section from your template.

Please share as much information as possible, so your Filebeat output section, the template and the ilm policy would be very useful

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 27, 2020, 4:23pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/10 "2020-05-27T16:23:55Z")

</div>

Hi Simon,

so the 1.-st step says "Create a lifecycle policy"

My policy "filebeat-cisco\_policy"

```
PUT _ilm/policy/filebeat-cisco_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "200gb"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "cold": {
        "min_age": "7d",
        "actions": {
          "set_priority": {
            "priority": 25
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

```

2-nd Create an index template  
I am using default Filebeat template with minor changes in "Index patterns" field

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7da3b38d66a3d6592a996b431c0ccaa9755a3657.png)

```
{
  "index": {
    "lifecycle": {
      "name": "filebeat-cisco_policy",
      "rollover_alias": "filebeat-cisco"
    },
    "mapping": {
      "total_fields": {
        "limit": "10000"
      }
    },
    "refresh_interval": "5s",
    "number_of_shards": "2",
    "query": {
      "default_field": [
        "message",
        "tags",
        "agent.ephemeral_id",
        "agent.id",
        "agent.name",
        "agent.type",
        "agent.version",
        "as.organization.name",
        "client.address",
        "client.as.organization.name",
        "client.domain",
        "client.geo.city_name",
        "client.geo.continent_name",
        "client.geo.country_iso_code",
        "client.geo.country_name",
        "client.geo.name",
        "client.geo.region_iso_code",
        "client.geo.region_name",
        "client.mac",
        "client.registered_domain",
        "client.top_level_domain",
        "client.user.domain",
        "client.user.email",
        "client.user.full_name",
        "client.user.group.domain",
        "client.user.group.id",
        "client.user.group.name",
        "client.user.hash",
        "client.user.id",
        "client.user.name",
        "cloud.account.id",
        "cloud.availability_zone",
        "cloud.instance.id",
        "cloud.instance.name",
        "cloud.machine.type",
        "cloud.provider",
        "cloud.region",
        "container.id",
        "container.image.name",
        "container.image.tag",
  ...deleted
        "fields.*"
      ]
    }
  }
}

```

3.-rd Bootstrap an index as my case it's done by filebeat

My Filebeat.yml

```
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["ip:9200"]
  username: "elastic"
  password: "password"
  ssl.certificate_authorities: ["/etc/elasticsearch/certs/ca/ca.crt"]
  ssl.certificate: "/etc/elasticsearch/certs/filebeat/filebeat.crt"
  ssl.key: "/etc/elasticsearch/certs/filebeat/filebeat.key"
 # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  indices:
    - index: "filebeat-netflow-%{+yyyy.MM.dd}-000001"
      when.equals:
        event.module: "netflow"

    - index: "filebeat-cisco-%{+yyyy.MM.dd}-000001"
      when.equals:
        event.module: "cisco"

```

I have deleted aliases from the template and I deleted the indices. When I got error again I will add error message.

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [May 28, 2020, 7:27am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/11 "2020-05-28T07:27:23Z")

</div>

There is still some issue.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/3/83fdf5fb8d7bf69371541f78805b70cf793759e9.png)

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/1/51972e89ed520288969221579311773aebb7d1c1.png)

![image](https://us1.discourse-cdn.com/elastic/original/3X/3/0/30e531b0f3d82af0f98f4371e86596a95626b7d9.png)

./log/elasticsearch/my-application.log

```
[2020-05-28T09:06:17,440][INFO][o.e.x.i.IndexLifecycleRunner] [node-1] policy [filebeat-netflow_policy] for index [filebeat-netflow-2020.05.28-000001] on an error step due to a transitive error, moving back to the failed step [check-rollover-ready] for execution. retry attempt [21]
[2020-05-28T09:06:17,443][INFO][o.e.x.i.IndexLifecycleRunner] [node-1] policy [filebeat-cisco_policy] for index [filebeat-cisco-2020.05.28-000001] on an error step due to a transitive error, moving back to the failed step [check-rollover-ready] for execution. retry attempt [27]
[2020-05-28T09:16:17,440][ERROR][o.e.x.i.IndexLifecycleRunner] [node-1] policy [filebeat-netflow_policy] for index [filebeat-netflow-2020.05.28-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [filebeat-netflow] does not point to index [filebeat-netflow-2020.05.28-000001]
        at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:104) [x-pack-core-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183) [x-pack-core-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211) [x-pack-core-7.7.0.jar:7.7.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]
[2020-05-28T09:16:17,443][ERROR][o.e.x.i.IndexLifecycleRunner] [node-1] policy [filebeat-cisco_policy] for index [filebeat-cisco-2020.05.28-000001] failed on step [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}]. Moving to ERROR step
java.lang.IllegalArgumentException: index.lifecycle.rollover_alias [filebeat-cisco] does not point to index [filebeat-cisco-2020.05.28-000001]
        at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:104) [x-pack-core-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:173) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:329) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:267) [x-pack-ilm-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:183) [x-pack-core-7.7.0.jar:7.7.0]
        at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:211) [x-pack-core-7.7.0.jar:7.7.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
        at java.lang.Thread.run(Thread.java:832) [?:?]

```

So how can I make It work?

Elasticsearch version is 7.7

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [June 1, 2020, 8:52am UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/12 "2020-06-01T08:52:39Z")

</div>

Any idea? I real would like to make this work.

---

<div class="post-metadata">

**Author:** ![Adriann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adriann/32/77780_2.png) [@Adriann](https://discuss.elastic.co/u/Adriann)\
**Post date:** [June 3, 2020, 6:16pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/13 "2020-06-03T18:16:06Z")

</div>

So basically after deleting rollover part in ilm policy it all started to work as I wanted

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 1, 2020, 6:16pm UTC](https://discuss.elastic.co/t/setting-different-ilm-policy-based-on-index-name/234497/14 "2020-07-01T18:16:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.