# SImple direct from log file JSON to elasticsearch

**URL:** <https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192>\
**Category:** Beats\
**Created:** [September 22, 2016, 1:51am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192 "2016-09-22T01:51:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brian\_G](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_g/32/44801_2.png) [@Brian\_G](https://discuss.elastic.co/u/Brian_G)\
**Post date:** [September 22, 2016, 1:51am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192/1 "2016-09-22T01:51:57Z")

</div>

Let's say I have code that is writing out tons of JSON documents to a log file. Consider these to be as is documents that should be in elasticsearch the same way they have been written to the log files. All I want to do then is efficiently/reliably ship those from the log file to Elasticsearch.

Is filebeat not built for that, since it adds all the filebeat meta data and just puts my document in the "message" field? Do you have to then send it off to logstash only to using a filter to get the original document and then off to elasticsearch? Am I missing something?

---

<div class="post-metadata">

**Author:** ![Vishnu\_Murty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vishnu_murty/32/12072_2.png) [@Vishnu\_Murty](https://discuss.elastic.co/u/Vishnu_Murty)\
**Post date:** [September 22, 2016, 5:53am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192/2 "2016-09-22T05:53:26Z")

</div>

We have used json\_lines codec in log stash input - [https://www.elastic.co/guide/en/logstash/2.4/plugins-codecs-json\_lines.html](https://www.elastic.co/guide/en/logstash/2.4/plugins-codecs-json_lines.html)

commands for reference : logstash -f logstash-configcollector.conf \< output\_collector.json

logstash-configcollector.conf contains :

input {  
stdin { codec =\> json\_lines  
} }

output {  
elasticsearch { hosts =\> ["localhost:9200"]}  
stdout { codec =\> rubydebug }  
}

output\_collector.json contains the json Data

Hope this may be helpfull..

Thanks  
Vishnu

---

<div class="post-metadata">

**Author:** ![Brian\_G](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brian_g/32/44801_2.png) [@Brian\_G](https://discuss.elastic.co/u/Brian_G)\
**Post date:** [September 22, 2016, 6:06am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192/3 "2016-09-22T06:06:13Z")

</div>

Thank you.

I see how to do it in logstash, but can this not be done directly with filebeat and avoiding logstash? The logstash seems like overkill.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 22, 2016, 6:08am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192/4 "2016-09-22T06:08:25Z")

</div>

The upcoming Filebeat 5 can deserialize lines of JSON text and ship the resulting objects.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 13, 2016, 1:51am UTC](https://discuss.elastic.co/t/simple-direct-from-log-file-json-to-elasticsearch/61192/5 "2016-10-13T01:51:59Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.