# How to send two different log file types from the same source directory to logstash **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [May 9, 2019, 12:53pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366 "2019-05-09T12:53:59Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mgolubov55](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@mgolubov55](https://discuss.elastic.co/u/mgolubov55)\ **Post date:** [May 9, 2019, 12:53pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/1 "2019-05-09T12:53:59Z")
I have been sending two different log files to logstash from two different source directories. One type is an "Index log" that is always named "IndexService\*.log", for example: IndexService-sv\_Toolkit-GBWLLSLO015-31012019-110628.293.log The other logs are not named as consistently. I have been using the following filebeat.yml configuration: filebeat.inputs: - type: log enabled: true paths: - C:\Logs\* tags: ["Reference","Jobs"] fields: document\_type: JobLogs multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after - type: log enabled: true paths: - C:\LogsIndexService\* tags: ["Reference","Indexes","Jobs"] fields: document\_type: IndexServiceLogsJob The specs on the job have just changed where both kind of logs need to be prospected from the same directory. Is there some kind of if/else logic I can apply to filter the files by filename and properly package them for logstash? ---
**Author:** ![Michal\_Pristas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michal_pristas/32/46639_2.png) [@Michal\_Pristas](https://discuss.elastic.co/u/Michal_Pristas)\ **Post date:** [May 9, 2019, 1:28pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/2 "2019-05-09T13:28:17Z")
hey @mgolubov55 you can use regex patterns in paths in a way to specify that IndexServiceLogs starts with index service other logs do not start with index service. for more capable parsing and condition you can use Logstash ---
**Author:** ![mgolubov55](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@mgolubov55](https://discuss.elastic.co/u/mgolubov55)\ **Post date:** [May 9, 2019, 4:55pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/3 "2019-05-09T16:55:49Z")
The following in filebeat.yml works for IndexService but not for nonIndexService. What am I doing wrong? - type: log enabled: true paths: - C:\Logs\IndexService-sv\_\*.log tags: ["Reference","Indexes","Jobs"] fields: document\_type: IndexServiceLogsJob - type: log enabled: true paths: - C:\Logs\'^?![IndexService-sv\_\*.log]' tags: ["Reference","Jobs"] fields: document\_type: JobLogs multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after ---
**Author:** ![mgolubov55](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@mgolubov55](https://discuss.elastic.co/u/mgolubov55)\ **Post date:** [May 9, 2019, 5:57pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/4 "2019-05-09T17:57:19Z")
I also tried the following - the IndexService log is read, but the other kind of log is ignored: - type: log enabled: true paths: - C:\Logs\IndexService-sv\_\*.log tags: ["Reference","Indexes","Jobs"] fields: document\_type: IndexServiceLogsJob - type: log enabled: true paths: - C:\Logs'^(?!IndexService-sv\_)\*.log' tags: ["Reference","Jobs"] fields: document\_type: JobLogs multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after ---
**Author:** ![martinr\_ubi](https://avatars.discourse-cdn.com/v4/letter/m/b5e925/32.png) [@martinr\_ubi](https://discuss.elastic.co/u/martinr_ubi)\ **Post date:** [May 9, 2019, 7:24pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/5 "2019-05-09T19:24:08Z")
Hi, path doesn’t take a regex but a go glob pattern. Here’s the link to an explanation of what that is: [https://golang.org/src/path/filepath/match.go?s=1226:1284#L34](https://golang.org/src/path/filepath/match.go?s=1226:1284#L34) ---
**Author:** ![mgolubov55](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@mgolubov55](https://discuss.elastic.co/u/mgolubov55)\ **Post date:** [May 10, 2019, 2:11pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/6 "2019-05-10T14:11:05Z")
Thank you, I found that out yesterday and revised my filebeat.yml accordingly and solved the issue: - type: log enabled: true paths: - C:\Logs\IndexService-sv\_\*.log tags: ["Reference","Indexes","Jobs"] fields: document\_type: IndexServiceLogsJob - type: log enabled: true paths: - C:\Logs[^IndexService]\*.log tags: ["Reference","Jobs"] fields: document\_type: JobLogs multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.negate: true multiline.match: after ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 7, 2019, 2:11pm UTC](https://discuss.elastic.co/t/how-to-send-two-different-log-file-types-from-the-same-source-directory-to-logstash/180366/7 "2019-06-07T14:11:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.