# ILM and using one index per Filebeat module

**URL:** <https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 29, 2019, 8:01pm UTC](https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752 "2019-10-29T20:01:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![philraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philraj/32/56809_2.png) [@philraj](https://discuss.elastic.co/u/philraj)\
**Post date:** [October 29, 2019, 8:01pm UTC](https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752/1 "2019-10-29T20:01:00Z")

</div>

Hi,

Is there any way (without disabling ILM) to have Filebeat use a different index for each enabled module? This was easy before ILM by adding conditions to the output.elasticsearch.indices:

```
setup.template.name: "logs"
setup.template.pattern: "logs-*"
output.elasticsearch:
    indices:
    - index: "logs-system-%{[agent.version]}-%{+yyyy.ww}"
      when.contains:
        fileset.module: system

```

But I'm having a very hard time finding information on how to accomplish something similar with ILM. Do I have to disable it to get this functionality?

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [October 29, 2019, 8:52pm UTC](https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752/2 "2019-10-29T20:52:03Z")

</div>

Ithink you can, I have Logstash using variables in the index name to write to different indices for each different value of that variable.

- you will probably have to have a template for each different index pattern (see example below)
- filebeat's setup probably won't do it
- you will have create the first index for each pattern before you write to it. [doc](https://www.elastic.co/guide/en/elasticsearch/reference/current/_apply_a_policy_to_a_create_index_request.html#_apply_a_policy_to_a_create_index_request)
- if you want the create date in the index name, do[this](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/indices-rollover-index.html#_using_date_math_with_the_rollover_api)

A sample ILM template. I use a higher priority, so it includes the default filebeat-%{{agent.version}} template, then adds ILM for this particular index pattern.

```
{
    "order" : 130,
    "index_patterns" : [
      "filebeat-7.3.2-zabbix-xx*"
    ],
    "settings" : {
      "index.lifecycle.name": "beat-retain-180",
      "index.lifecycle.rollover_alias": "filebeat-7.3.2-zabbix-xx"
    }
}

```

Sample output from logstash:

```
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{[fields][app_id]}-%{[fields][campus]}"

```

This example if apache logs from a zabbix server. Filebeat uses the fields to set these variables:

```
fields:
  campus: xx
  app_id: zabbix
```

---

<div class="post-metadata">

**Author:** ![philraj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philraj/32/56809_2.png) [@philraj](https://discuss.elastic.co/u/philraj)\
**Post date:** [October 31, 2019, 7:42pm UTC](https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752/3 "2019-10-31T19:42:29Z")

</div>

I'll take a look at this, thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2019, 7:42pm UTC](https://discuss.elastic.co/t/ilm-and-using-one-index-per-filebeat-module/205752/4 "2019-11-28T19:42:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.