# Filebeat with multiple log path should direct to different Index and Should follow ILM,ILM policy and rollover already defined in elastic **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 22, 2023, 11:17am UTC](https://discuss.elastic.co/t/filebeat-with-multiple-log-path-should-direct-to-different-index-and-should-follow-ilm-ilm-policy-and-rollover-already-defined-in-elastic/328236 "2023-03-22T11:17:03Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [March 22, 2023, 11:17am UTC](https://discuss.elastic.co/t/filebeat-with-multiple-log-path-should-direct-to-different-index-and-should-follow-ilm-ilm-policy-and-rollover-already-defined-in-elastic/328236/1 "2023-03-22T11:17:03Z")
Hello All, I've a requirement wherein I would like to have single filebeat.yml and this will have different log paths and will direct the data to respective diffrent index according to path. ```auto Now this filebeat.yml would be installed in various servers with installed filebeat service and then all the log path that are same in all servers should always write data to current rollover index. ``` ex:path 1 logs-\>direct to-\>mis-log index ex:path 2 logs-\>direct to-\>mis-monitoring-usecase index etc In my index template already I've defined the ILM and rollover policy. TEMPLATE: ```auto PUT _index_template/mis-monitoring-usecases { "template": { "settings": { "index": { "lifecycle": { "name": "mis-monitoring-common-policy", "rollover_alias": "mis-monitoring-usecases" }, "default_pipeline": "mis-usecases-ingest-pipeline", "number_of_shards": "1", "number_of_replicas": "0" } }, "mappings": { "properties": { } } }, "index_patterns": [ "mis-monitoring-usecases-*" ], "composed_of": [] } mis-monitoring-common- ``` Current filebeat.yml:(for two log paths direct to two seperate index,but dont follow rollover policy/pattern) Current filebeat.yml: ```auto filebeat.inputs: - type: log enabled: true paths: - /k/cpp/LOG_new/CMS/**/*.log fields: index: mis-monitoring-usecases ignore_older: 1h include_lines: ['UsecaseMonitoring.*\|'] multiline.type: pattern multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z multiline.negate: true multiline.match: after scan_frequency: 30s harvester_limit: 100 close_inactive: 30m close_removed: true clean_removed: true processors: - add_host_metadata: null - drop_fields: when: equals: agent.type: filebeat fields: - agent.hostname - agent.id - agent.type - agent.ephemeral_id - agent.version - log.offset - log.flags - input.type - ecs.version - host - type: log enabled: true paths: - /L/app/LOG_LOOT/Demo/*.log fields: index: mis-log ignore_older: 1h include_lines: - ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z ?(.*) multiline.type: pattern multiline.pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z multiline.negate: true multiline.match: after scan_frequency: 30s harvester_limit: 100 close_inactive: 30m close_removed: true clean_removed: true reload.enabled: false processors: - add_host_metadata: null - drop_fields: when: equals: agent.type: filebeat fields: - agent.hostname - agent.id - agent.type - agent.ephemeral_id - agent.version - log.offset - log.flags - input.type - ecs.version - host.os - host.id - host.mac - host.architecture filebeat.config.modules: path: ${path.config}/modules.d/*.yml setup.ilm.enabled: true setup.template.enabled: false setup.ilm.pattern: '{now/d}-000001' setup.kibana: host: http://abc:5601 output.elasticsearch: hosts: - http://abc:9200 index: "%{[fields.index]}" monitoring.enabled: true monitoring.elasticsearch: null ``` CHALLENGE: The ILM is not followed i.e ideally it should make two index of this pattern,which is not created: mis-monitoring-usecases-2023.03.22-000001 mis-log-2023.03.22-000001 Read documentation its says: When [index lifecycle management (ILM)](https://www.elastic.co/guide/en/beats/filebeat/7.9/ilm.html) is enabled, the default `index` is `"filebeat-%{[agent.version]}-%{+yyyy.MM.dd}-%{index_num}"` , for example, `"filebeat-7.9.3-2023-03-09-000001"` . Custom `index` settings are ignored when ILM is enabled. If you’re sending events to a cluster that supports index lifecycle management, see [_Index lifecycle management (ILM)_](https://www.elastic.co/guide/en/beats/filebeat/7.9/ilm.html) to learn how to change the index name. > **[Configure the Elasticsearch output | Filebeat Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.9/elasticsearch-output.html)** If I wanted to create my own index name, I can't use the ILM anymore? ,what are the alternatives ,sending filebaet to logstash is not an option for me. elk 7.9.1 version Kindly suggest Thanx ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 19, 2023, 1:17pm UTC](https://discuss.elastic.co/t/filebeat-with-multiple-log-path-should-direct-to-different-index-and-should-follow-ilm-ilm-policy-and-rollover-already-defined-in-elastic/328236/2 "2023-04-19T13:17:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.