# Elastic Json parse into logstash **URL:** \ **Category:** Logstash\ **Created:** [October 10, 2018, 11:58am UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840 "2018-10-10T11:58:43Z")\ **Posts on this page:** 11\ **Page:** 2
**Author:** ![moughrom](https://avatars.discourse-cdn.com/v4/letter/m/71c47a/32.png) [@moughrom](https://discuss.elastic.co/u/moughrom)\ **Post date:** [October 25, 2018, 8:55am UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/21 "2018-10-25T08:55:05Z")
Hey Dominic, I have not a big experience on tools like AWK or others. please advice tools that I may use instead of AWK. Best and thanks a lot for your support. Mohamed ---
**Author:** ![Dominic\_Page](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominic_page/32/48318_2.png) [@Dominic\_Page](https://discuss.elastic.co/u/Dominic_Page)\ **Post date:** [October 25, 2018, 12:01pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/22 "2018-10-25T12:01:48Z")
Hi @moughrom Your choice of technology to manipulate JSON is up to you. If Logstash were connecting to Elasticsearch via Elastic Security, then the Elasticsearch input plugin for Logstash would already be receiving JSON, off-the-shelf, and you would not have this issue. As you have a different security model, you are extracting the documents as JSON documents within a larger JSON document. You need a tool that will allow you to parse those documents into a format that can be easily read by Logstash, for example a file containing multiple, single JSON documents, each on a separate line. As well as the standard command line tools, there are many scripting languages and open-source tools available, I hope you can find one that suits your need. ---
**Author:** ![moughrom](https://avatars.discourse-cdn.com/v4/letter/m/71c47a/32.png) [@moughrom](https://discuss.elastic.co/u/moughrom)\ **Post date:** [October 25, 2018, 2:23pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/23 "2018-10-25T14:23:22Z")
elastic security is install on my cluster (6.4) but got issue with json multiline. can you name few tools you are talking about. ---
**Author:** ![Dominic\_Page](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominic_page/32/48318_2.png) [@Dominic\_Page](https://discuss.elastic.co/u/Dominic_Page)\ **Post date:** [October 25, 2018, 2:46pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/24 "2018-10-25T14:46:42Z")
Hi @moughrom > [@moughrom](#): > > elastic security is install on my cluster (6.4) If you have Elastic Security Installed, your Logstash should be able to connect directly to Elasticsearch. Then Logstash can read the documents directly and there will be no need for parsing JSON. Logstash can make the necessary updates and additions and then index the amended data to a new index in Elasticsearch, again securely. You can find guidance on connecting Logstash to Elastic Security here: > **[Secure your connection to Elasticsearch | Logstash Reference \[8.11\] | Elastic](https://www.elastic.co/guide/en/logstash/current/ls-security.html)** Hope this helps ---
**Author:** ![Dominic\_Page](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominic_page/32/48318_2.png) [@Dominic\_Page](https://discuss.elastic.co/u/Dominic_Page)\ **Post date:** [October 25, 2018, 2:51pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/25 "2018-10-25T14:51:34Z")
Hi @moughrom > [@moughrom](#): > > can you name few tools you are talking about. I'm sorry, I can't make recommendations about third party tools ---
**Author:** ![moughrom](https://avatars.discourse-cdn.com/v4/letter/m/71c47a/32.png) [@moughrom](https://discuss.elastic.co/u/moughrom)\ **Post date:** [October 25, 2018, 3:09pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/26 "2018-10-25T15:09:01Z")
Thanks Dominic, I have googled "json tools linux command line" and got a bunch of github projects. My issue at the end was the grep/awk/sed was getting out after the first event and not processing next events. I have started learning awk/grep/sed to understand, but as an expert cans you see why we got this behavior, from my experience in powershell when a pipeline end is that a part of the pipeline stop sending to the next pipeline and display on the screen instead and breaking the chains. please advice. ---
**Author:** ![moughrom](https://avatars.discourse-cdn.com/v4/letter/m/71c47a/32.png) [@moughrom](https://discuss.elastic.co/u/moughrom)\ **Post date:** [October 25, 2018, 3:12pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/27 "2018-10-25T15:12:09Z")
curl -X GET [http://0.0.0.0:9200/index-2018-08-09/\_search](http://0.0.0.0:9200/index-2018-08-09/_search) | sed -e "s/^.\*\_source"://" | grep -v "}]}}" | awk '/^}/ {print (NR==1?"":RS)$0;next} {printf "%s",$0}' | grep -v "^}$" | sed -e "s/$/}/" **\>\>** /Users/Shared/logs/test.json ---
**Author:** ![Dominic\_Page](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominic_page/32/48318_2.png) [@Dominic\_Page](https://discuss.elastic.co/u/Dominic_Page)\ **Post date:** [October 25, 2018, 3:22pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/28 "2018-10-25T15:22:39Z")
Hi @moughrom Are you using bash or powershell? ---
**Author:** ![Dominic\_Page](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dominic_page/32/48318_2.png) [@Dominic\_Page](https://discuss.elastic.co/u/Dominic_Page)\ **Post date:** [October 25, 2018, 3:28pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/29 "2018-10-25T15:28:56Z")
@moughrom I gave a bash example to show that in principle logstash could be used to ingest the data once this had been prepared. As mentioned above I do not have the resources to replicate your full environment, so it will not be possible for me to progress your JSON parsing issue further. I recommend you seek a local resource who can help you with your parsing issue hands-on. ---
**Author:** ![moughrom](https://avatars.discourse-cdn.com/v4/letter/m/71c47a/32.png) [@moughrom](https://discuss.elastic.co/u/moughrom)\ **Post date:** [October 25, 2018, 5:36pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/30 "2018-10-25T17:36:26Z")
Many Thanks for your answers. it has really help. Best. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 22, 2018, 5:36pm UTC](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840/31 "2018-11-22T17:36:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/elastic-json-parse-into-logstash/151840.md?page=1)