# Regarding reading the filebeat output to a file as input to logstash

**URL:** <https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 2, 2016, 1:50pm UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955 "2016-05-02T13:50:49Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)\
**Post date:** [May 2, 2016, 1:50pm UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/1 "2016-05-02T13:50:49Z")

</div>

when we store filebeat data to logstash and apply filters and then ouput into a file & again pass file data as input to logstash and then to elasticsearch, all the fields will be merged into one field w.

How can we read the filebeat out written to a file from logstash correctly. There is some firewall issue dure to which we cannot directly write data from filebeat to logstash and then to elastic search.

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [May 3, 2016, 7:09pm UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/2 "2016-05-03T19:09:25Z")

</div>

So your setup is: FB -\> LS -\> File -\> LS -\> ES ?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 3, 2016, 8:11pm UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/3 "2016-05-03T20:11:43Z")

</div>

It sounds like Filebeat is writing lines with JSON to a file and you're not using the json codec for the file input that reads that file.

---

<div class="post-metadata">

**Author:** ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)\
**Post date:** [May 4, 2016, 3:34am UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/4 "2016-05-04T03:34:32Z")

</div>

Yes, Correcte the set is FB -\> LS -\> File -\> LS -\> .ES

We tried using json code in file input but still it's not getting the fields correctly.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 4, 2016, 5:30am UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/5 "2016-05-04T05:30:01Z")

</div>

There are now two threads discussing exactly the same problem (this one and [Regarding Filebeat Ouput to a File in Remote Server or NFS](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/27)). Please pick one thread and stick to it.

---

<div class="post-metadata">

**Author:** ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)\
**Post date:** [May 4, 2016, 5:31am UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/6 "2016-05-04T05:31:44Z")

</div>

Hi,

We have added code=json in the file input plugin and run logstash , But again the ouput is showing in the fillowing format.

["message": "{"message":"root 19712 19709 0 02:08 ttyS0 00:00:00 grep dtpd","@timestamp":"2016-05-02T13:36:30.654Z","offset":366,"source":"/var/log/broncos\_logs/dvt\_1.txt","chs":"test","date":"02-05-2016"}"

Okay i'll stick to this thread itself may be i'll attache the other one to this.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 4, 2016, 5:32am UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/7 "2016-05-04T05:32:57Z")

</div>

Please answer all the questions I asked in the other thread.

---

<div class="post-metadata">

**Author:** ![vivc](https://avatars.discourse-cdn.com/v4/letter/v/ce7236/32.png) [@vivc](https://discuss.elastic.co/u/vivc)\
**Post date:** [May 4, 2016, 5:34am UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/8 "2016-05-04T05:34:00Z")

</div>

I mean i'll update on the thread [Regarding Filebeat Ouput to a File in Remote Server or NFS](https://discuss.elastic.co/t/regarding-filebeat-ouput-to-a-file-in-remote-server-or-nfs/46038/27)). I'll check how to attach this thread to the other one.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:52pm UTC](https://discuss.elastic.co/t/regarding-reading-the-filebeat-output-to-a-file-as-input-to-logstash/48955/9 "2017-07-05T21:52:25Z")

</div>


