# Filebeat for both system|service logs and application logs **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [July 10, 2019, 11:36am UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742 "2019-07-10T11:36:04Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![coudenysj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/coudenysj/32/23108_2.png) [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Post date:** [July 10, 2019, 11:36am UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/1 "2019-07-10T11:36:05Z")
Hi, I'm having trouble to pick a solution to use filebeat for both the default system|service log files and application log files. I'm logging my application logs in json, so not much processing must be done, but I would like to store the application logs in a different index (instead of filebeat-\*) because a lot of application log fields are not available in the filebeat ECS list. Should I just add those fields to the filebeat template? Or should I start a different filebeat process to ship those logs to the different index (because filebeat cannot output to different indexes)? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [July 10, 2019, 2:29pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/2 "2019-07-10T14:29:46Z")
Filebeat can output to different indices. But you'd need to setup the template for other indices yourself (plus you'd might have to disable ILM). e.g. ```auto filebeat.inputs: - input: log ... fields.index: applog // output to - input: log ... output.elasticsearch: index: '%{[fields.index}:filebeat}-%{[agent.version}}-%{+yyyy.MM.dd}' ``` ---
**Author:** ![coudenysj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/coudenysj/32/23108_2.png) [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Post date:** [July 11, 2019, 12:57pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/3 "2019-07-11T12:57:15Z")
Thanks for the feedback! It looks like my setup ignores the `output.elasticsearch.index` setting. I had to enable specify the `setup.template.name` and `setup.template.pattern` (which I gave the default value), but filebeat is not creating the different indices. I used this value (just to be sure it was changing): `index: "%{[fields.index]:filebeat}-%{[agent.version]}---%{+yyyy.MM.dd}"` I restarted filebeat multiple times, but nothing is changing (it keeps writing to the `filebeat-7.2.0-2019.07.11-000001` index). ---
**Author:** ![coudenysj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/coudenysj/32/23108_2.png) [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Post date:** [July 11, 2019, 2:09pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/4 "2019-07-11T14:09:29Z")
I've got it working like this: ```auto indices: - index: "%{[fields.index]:filebeat}-%{[agent.version]}-%{+yyyy.MM.dd}" ``` But the normal `index` approach wasn't doing much, probably a bug? ---
**Author:** ![coudenysj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/coudenysj/32/23108_2.png) [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Post date:** [July 11, 2019, 2:27pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/5 "2019-07-11T14:27:37Z")
The problem now, is that the ILM is acting crazy. My filebeat logs aren't written into the -00001 index. Is this something I can fix? ![DeepinScreenshot_select-area_20190711162614](https://us1.discourse-cdn.com/elastic/original/3X/f/4/f4b2b22ac8158fde55c2cbc59130eb1cda57c1d7.png) ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [July 12, 2019, 1:15pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/6 "2019-07-12T13:15:18Z")
When using the `index` setting you have to disable ILM. Did you disable ILM? When using ILM one has to use a write alias. By default the write alias is `filebeat-7.2.0`. This alias creates indices named `filebeat-7.2.0--0000001`. You have one index with and one index without ILM configured. If you want to use custom indices with ILM, then you will need to setup ILM yourself and configure `index` such the Beats will write to the write alias you've setup beforehand. ---
**Author:** ![coudenysj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/coudenysj/32/23108_2.png) [@coudenysj](https://discuss.elastic.co/u/coudenysj)\ **Post date:** [July 17, 2019, 6:49am UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/7 "2019-07-17T06:49:17Z")
Thanks for the reply @steffens! So I'm probably better of just indexing my custom data (which also follows the ECS as much as possible) in the default filebeat indices? That way the ILM is working as expected and the overhead of the filebeat mapping will be minimal, correct? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [July 17, 2019, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/8 "2019-07-17T12:00:13Z")
> So I'm probably better of just indexing my custom data (which also follows the ECS as much as possible) in the default filebeat indices? > > That way the ILM is working as expected and the overhead of the filebeat mapping will be minimal, correct? Well, as always it depends 🙂 But I think I would agree here. If you are unsure, add some meta-data to your events. This would allow you to filter and reindex said events in the future, in case you want to change something. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 14, 2019, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-for-both-system-service-logs-and-application-logs/189742/9 "2019-08-14T12:00:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.