# Reading multiple files and writing into multiple files

**URL:** <https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 6, 2018, 11:24am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150 "2018-08-06T11:24:38Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\
**Post date:** [August 6, 2018, 11:24am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150/1 "2018-08-06T11:24:39Z")

</div>

I want to read multiple log files and write into multiple files (intention is to test). My Filebeat configuration looks like below :

```
filebeat.prospectors:

- input_type: log
  enabled: true
  paths:
    - C:\data\Logs\eis.log
  fields: {log_type: eis}

- input_type: log
  enabled: true
  paths:
    - C:\data\Logs\sa.log
  fields: {log_type: sa}
  
  multiline.match: after
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

output.file:
  path: "c:/var/"
  filename: "filebeat.log"
```

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 7, 2018, 5:26pm UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150/2 "2018-08-07T17:26:10Z")

</div>

Hi @Raghuveer_SJ,

Beats can only have an output, and the [file output](https://www.elastic.co/guide/en/beats/filebeat/current/file-output.html) can only have a file.

---

<div class="post-metadata">

**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\
**Post date:** [August 8, 2018, 3:44am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150/3 "2018-08-08T03:44:42Z")

</div>

But I see filebeat has written both the files into a single file. On console it only says harvesting with one of the filename. So I was a bit confused. However I see both the files are read so its fine.

but I don't see the stacktrace being grouped by filebeat, its sent as individual lines. So I tried adding the codec in logstash as :

```
input {

beats {
	port=>5044
	 codec => multiline {
                  pattern => "^%{TIMESTAMP_ISO8601} "
                  negate => true
                  what => previous
               }
}
}

```

I see logstash says filebeat does not support this configuration. Please suggest.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [August 8, 2018, 9:50am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150/4 "2018-08-08T09:50:17Z")

</div>

You should add the multiline configuration in filebeat itself, check the documentation for filebeat, it is very similar to the one in logstash: [https://www.elastic.co/guide/en/beats/filebeat/6.3/multiline-examples.html](https://www.elastic.co/guide/en/beats/filebeat/6.3/multiline-examples.html)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2018, 9:50am UTC](https://discuss.elastic.co/t/reading-multiple-files-and-writing-into-multiple-files/143150/5 "2018-09-05T09:50:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.