# Specify ILM policy for Multiple Elasticsearch outputs
**URL:** https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129
**Category:** Beats
**Tags:** filebeat
**Created:** [November 15, 2019, 7:49pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129 "2019-11-15T19:49:14Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@TonyLuc](https://discuss.elastic.co/u/TonyLuc)
#### Post date: [November 15, 2019, 7:49pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/1 "2019-11-15T19:49:14Z")
_Scenario_
I am using filebeat to collect logs from 2 different sources.
It is going to 2 different indexes. Thus in the filebeat.yml the following are added to the Elasticsearch output:
output.elasticsearch:
hosts: ["someserver:9200"]
indices:
- index: "iis-%{[agent.version]}-%{+yyyy.MM.dd}"
when.equals:
event.module: "iis"
index: "logs-%{[agent.version]}-%{+yyyy.MM.dd}"
I have switch off
setup.ilm.enabled: false
My intentions (using the above example) is to save any logs from IIS to indices starting with “iis-”, while the defaults go to indices starting with “logs-”.
My question is: I want to use different ILM policies for different indices (the retention period will be different), how to I associated the ILM policy name to each index?
Any advises?
Thanks!
---
### Author:  [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [November 17, 2019, 1:29am UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/2 "2019-11-17T01:29:56Z")
Create a template for each index with the desired ILM policy.
It looks like you are going to create daily indices, so aren't using ILM for rollover. If you were, it's a little more complicated 🙂
---
### Author:  [@TonyLuc](https://discuss.elastic.co/u/TonyLuc)
#### Post date: [November 18, 2019, 10:49pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/3 "2019-11-18T22:49:08Z")
I am just using ILM just for deleting indices after a certain period.
The behavior is based on version 7.4.2
I have thought about using the template but I have problem loading the initial template.
Let me explain, when you setup a filebeat agent using IIS module and specified the template and index name and having only one elasticsearch.output as follow:
setup.template.name: "baa-%{[agent.version]}"
setup.template.pattern: "baa-%{[agent.version]}-\*"
hosts: ["blacksheep:9200"]
index: "baa-%{[agent.version]}-%{+yyyy.MM.dd}"
ElasticSearch will automatically create a template called “baa-7.4.2”.
Question here is how I load this manually? This is first required to allow me to add the ILM policy name.
In other observations
I have updated from version 7.3.2 to 7.4.2, the following behaved differently.
In a multiple elastic outputs scenario as follows:
setup.template.name: "iis-%{[agent.version]}"
setup.template.pattern: "iis-%{[agent.version]}-\*"
setup.template.name: "logs-%{[agent.version]}"
setup.template.pattern: "logs-%{[agent.version]}-\*"
output.elasticsearch:
hosts: ["someserver:9200"]
indices:
- index: "iis-%{[agent.version]}-%{+yyyy.MM.dd}"
when.equals:
event.module: "iis"
index: "logs-%{[agent.version]}-%{+yyyy.MM.dd}"
In version 7.3.2 both templates iis.7.3.2 and logs.7.3.2 were created.
In version 7.4.2 only template logs.7.3.2 was created.
Did anyone notice this?
---
### Author:  [@rugenl](https://discuss.elastic.co/u/rugenl)
#### Post date: [November 19, 2019, 3:44pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/4 "2019-11-19T15:44:45Z")
Since we use Logstash, I don't use the beats setup after initial setup. (I don't what to give the access necessary to manage templates to things that are just sending logs)
My method.... I setup the default template for each version, like "filebeat-7.4.0", by default, it has Order:1. We name our indices so that they match this index pattern, with suffixes to separate as needed, for example "filebeat-7.4.0-thing1-campus1".
If I needed a template "baa-7.4.0-\*", I would just GET the default template, edit it, and POST it back using devtools or curl, but I keep the ILM part out of these templates as below.
I can create a template for index patterns "filebeat\*thing1" to add mappings and I use order: 100 to trump any future field name conflicts from Elastic. This template doesn't repeat anything from the default template.
I can also create a template for index patterns "\*-campus1" to add ILM options for those with order:150.
The beats setup changes from 6.x to 7.x are confusing. I'm not using 7.4.2 yet, so maybe they get more confusing. That and using Lostash are reasons why I just did it myself :-).
Good luck
---
### Author:  [@TonyLuc](https://discuss.elastic.co/u/TonyLuc)
#### Post date: [November 19, 2019, 5:36pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/5 "2019-11-19T17:36:35Z")
Thanks for the helps, Len.
1. I am using yours suggestion by first exporting the default template from filebeat.
Reference: [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html)
Powershell script: .\filebeat.exe export template --es.version 7.4.2 | Out-File -Encoding UTF8 filebeat.template.json
1. Add the policy name to the JSON file.
2. Post it to Elastic as a template “baa-7.4.2”
3. Make sure setup.template.overwrite: false. I been burnt too many times, I find it less of a headache to set it explicitly than relying on the default.
4. Set my index as index: "baa-%{[agent.version]}-%{+yyyy.MM.dd}"
My life should be good till I have to remember to repeat it again for the future upgrades.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 17, 2019, 5:36pm UTC](https://discuss.elastic.co/t/specify-ilm-policy-for-multiple-elasticsearch-outputs/208129/6 "2019-12-17T17:36:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.