# Are Beats designed to be used with ILM?

**URL:** <https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168>\
**Category:** Beats\
**Tags:** docker, ilm-index-lifecycle-management, filebeat, metricbeat\
**Created:** [July 8, 2021, 9:59am UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168 "2021-07-08T09:59:37Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kodka](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@kodka](https://discuss.elastic.co/u/kodka)\
**Post date:** [July 8, 2021, 9:59am UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/1 "2021-07-08T09:59:37Z")

</div>

I'm trying to implement ILM to Filebeat indices, but this popped up in the process:

We have different Filebeat versions and multiple modules like docker, access, and many others, but for simplicity, let's take a look at docker only.

We use labels for each Filebeat module, so we can differentiate them in Logstash and separate them in different indices, for example in Logstash we do - if label == docker, index =\> [metadata-beat]-[metadata-version]-docker

In this way, we will have indices:  
filebeat-6.8.0-docker  
filebeat-7.6.0-docker  
filebeat-7.11.1-docker

With this setup, having different index templates for each version is not a problem, since they match index pattern filebeat-version-\*, **but with ILM we don't have a way to add Filebeat version in the name dynamically.**

Seems like the only way is to manually create rollover index for each filebeat version and module, but that will be a lot of work, since we have other beats with the same logic, and i will need to repeat the process for every new version that come.

Is there a way to achieve this in a more automated manner?

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [July 9, 2021, 9:06am UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/2 "2021-07-09T09:06:02Z")

</div>

I know that there is the setup pattern property: [Configure index lifecycle management | Filebeat Reference [7.13] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/ilm.html#setup-ilm-pattern-option) . Not sure if it helps you with your case.

---

<div class="post-metadata">

**Author:** ![kodka](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@kodka](https://discuss.elastic.co/u/kodka)\
**Post date:** [July 12, 2021, 10:48am UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/3 "2021-07-12T10:48:54Z")

</div>

Thanks, that will be useful! Is there a way to combine this features with option to output each module to different index dynamically?  
Something like - `setup.ilm.pattern: "%{[event][module]}-{now/d}-000001"` (this one did not work)

---

<div class="post-metadata">

**Author:** ![kodka](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@kodka](https://discuss.elastic.co/u/kodka)\
**Post date:** [July 12, 2021, 1:16pm UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/4 "2021-07-12T13:16:07Z")

</div>

I tried with  
`setup.ilm.rollover_alias: "metricbeat-%{[agent.version]}-%{[service.type]}"`  
as well, no success. It creates template at the start of the service, so maybe it's not desighned to be dynamic.

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [July 13, 2021, 1:02pm UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/5 "2021-07-13T13:02:08Z")

</div>

Correct. You'd have to either manually create the policies and templates and then u can dynamically send the data to the respective alias using the conditional settings on the elasticsearch output.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 10, 2021, 3:02pm UTC](https://discuss.elastic.co/t/are-beats-designed-to-be-used-with-ilm/278168/6 "2021-08-10T15:02:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.