# Trouble with Logstash JSON parsing **URL:** \ **Category:** Logstash\ **Created:** [November 13, 2017, 4:04pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426 "2017-11-13T16:04:01Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 13, 2017, 4:04pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/1 "2017-11-13T16:04:01Z")
I'm calling an API from a script. The script then runs the results through "jq" to rearrange the fields properly which then outputs to a .json file. In Logstash I'm using File input to read the file and trying to output to Elasticsearch. I've run the json file through jsonlint and it is valid but I'm receiving the following error when I output to stdout: > [logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>" }", :exception=\>#\ [ > { > "id": "7744", > "canUpdate": true, > "canDelete": true, > "canArchive": true, > "info": [{ > "problem\_type": null, > "Category": "" > }, > { > "closure\_information": 0, > "Closure Information": "" > }, > { > "insert\_time": 1509992459000, > "Request time": "2017-11-06 13:20:59.0" > } > ] > }, > { > "id": "47", > "canUpdate": true, > "canDelete": true, > "canArchive": true, > "info": [{ > "problem\_type": "PAN", > "Category": "PAN" > }, > { > "closure\_information": 1, > "Closure Information": "Solved (Permanently)" > }, > { > "insert\_time": 1466446314000, > "Request time": "2016-06-20 14:11:54.0" > } > ] > } > ] ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 14, 2017, 6:28am UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/2 "2017-11-14T06:28:52Z")
The file input reads files line by line. If you want to slurp an entire file into a single event you need to use a multiline codec. I'm sure examples have been posted in the past. ---
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 14, 2017, 2:47pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/3 "2017-11-14T14:47:07Z")
Magnus - Thank you for your input. I certainly have seen examples for multiline codec. Will check that out. Thank you. ---
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 14, 2017, 3:04pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/4 "2017-11-14T15:04:25Z")
Since the output file that is read is actually just created from a bash script could I just simply run the script and pipe results directly to Logstash? This would negate the need to create and read a separate file. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 14, 2017, 3:08pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/5 "2017-11-14T15:08:46Z")
Sure, you could do that but I don't recommend it. - If Logstash hangs (e.g. because its outputs are unavailable) your bash script will also hang if it produces a lot of data and fills its pipe buffer. - Logstash can't be restarted while the script is running. ---
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 14, 2017, 3:49pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/6 "2017-11-14T15:49:32Z")
Makes sense. I may just look at pushing directly to Elasticsearch then. Thanks. ---
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 14, 2017, 4:27pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/7 "2017-11-14T16:27:32Z")
Pushing to Elasticsearch worked BUT the entire array went in as one single document. What I need is each "id" separately. I originally tried the EXEC logstash input which used CAT on the file. The output from this looked perfect meaning each "ID" with its associated information on a separate line. In your opinion might this be a better approach? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 14, 2017, 7:30pm UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/8 "2017-11-14T19:30:01Z")
Sure, using an exec input is probably more straightforward than a file input. Of course, you don't get any tracking of what data you've processed but that's perhaps not a problem. ---
**Author:** ![tuxnader](https://avatars.discourse-cdn.com/v4/letter/t/aca169/32.png) [@tuxnader](https://discuss.elastic.co/u/tuxnader)\ **Post date:** [November 22, 2017, 1:26am UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/9 "2017-11-22T01:26:16Z")
After further examining the output looks like the it's "pretty printed" with a newline character (\n) after each and every line including those with just a "[" and "{". What would be the best way to deal with this? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 22, 2017, 6:17am UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/10 "2017-11-22T06:17:42Z")
If you use an exec input that's not a problem and if you use a file input you need to use a multiline codec as I said earlier. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 20, 2017, 6:32am UTC](https://discuss.elastic.co/t/trouble-with-logstash-json-parsing/107426/11 "2017-12-20T06:32:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.