# Filebeat read different log paths and write to different elastic index with their own ilm policy,How? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 14, 2023, 9:10am UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638 "2023-03-14T09:10:21Z")\ **Posts on this page:** 7\ **Page:** 2
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [March 22, 2023, 1:12pm UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/21 "2023-03-22T13:12:59Z")
Yes,I did same , but my index is not getting created with above filebeat.yml in which I've added below also. index: "%{[fields.index]}-%{+yyyy.MM.dd}-000001". ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 22, 2023, 1:24pm UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/22 "2023-03-22T13:24:26Z")
> [@PRASHANT\_MEHTA](#): > > Yes,I did same , but my index is not getting created with above filebeat.yml in which I've added below also. > index: "%{[fields.index]}-%{+yyyy.MM.dd}-000001". Yeah, but I don't think this is correct, if you are using rollover, you need to point to the rollover alias, if it is not working while pointing to the rollover alias than it may be another issue. Unfortunatelly I can not help further as I do not use filebeat directly to elasticsearch and I also do not use rollover, my only recommendation would be to check the documentation and validate each step of the configuration. ---
**Author:** ![Jason\_Paralta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_paralta/32/118856_2.png) [@Jason\_Paralta](https://discuss.elastic.co/u/Jason_Paralta)\ **Post date:** [March 23, 2023, 6:53pm UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/23 "2023-03-23T18:53:39Z")
Hi @PRASHANT_MEHTA, I suppose, I understand your intention. If I'm not wrong you are trying to achieve from filebeat.yml send two types of logs in two separate index and you have defined ILM and rollover alias in index template. Now you want this data to be written to their respective index with their rollover pattern. I suppose this won't be possible through filebeat and you need to use logstash to achieve this. I am curious to know if you have achieved this through filebeat. Through documentation it seem not possible as you've self declared indexes. ---
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [March 24, 2023, 9:34am UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/24 "2023-03-24T09:34:26Z")
Yes, you got it correct and I'm still looking out ways to do without logstash.I can't use logstash. @legoguy1000 @Marius_Iversen you've any advice what I'm doing wrong ? Thanx ---
**Author:** ![jba](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jba/32/118482_2.png) [@jba](https://discuss.elastic.co/u/jba)\ **Post date:** [March 30, 2023, 9:52pm UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/25 "2023-03-30T21:52:05Z")
Big warning: I only have experience with Filebeat -\> Logstash -\> Elasticsearch. I **think** what you can do is to just have Filebeat send the data to Elasticsearch. On/In Elasticsearch you can define/update something called an Ingest Pipeline. I think you should already have a simple Ingest Pipeline called `main` defined there (it should be visible via Kibana -\> Stack Management). I **think** you can modify this pipleline to do some manipulation of the data coming in - like separating the data into different indices. But I have never tried it. ---
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [April 11, 2023, 9:47am UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/26 "2023-04-11T09:47:23Z")
Hello @jba , Thanx for your suggestion on this and time to look into this.The way you recommend didn't helped. I have simple requirement : 1)different log paths 2)send the data of diffrent log path to diffrent indices using filebeat. 3)for every log path its index template with ilm and rollover is defined and log ingest pipeleine also written[grok pattern] 4)Somehow the data don't go to elastic index and dosent follow the ilm policy I'm not sure how it can be done by using only filebeat and not using logstash. Thanx ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 9, 2023, 9:48am UTC](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638/27 "2023-05-09T09:48:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/filebeat-read-different-log-paths-and-write-to-different-elastic-index-with-their-own-ilm-policy-how/327638.md?page=1)