# Varying amount of JSON parse errors when parsing the same files **URL:** \ **Category:** Logstash\ **Created:** [June 28, 2018, 9:56pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864 "2018-06-28T21:56:12Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Matjo](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@Matjo](https://discuss.elastic.co/u/Matjo)\ **Post date:** [June 28, 2018, 9:56pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/1 "2018-06-28T21:56:12Z")
I have a very simple setup: ``` input { file { path => "/home/elasticadmin/azure/**/*.*" codec => "json" close_older => 5 } } filter {} output { elasticsearch { user => "user" password => "pwd" index => "name" hosts => ["localhost:9200"] } } ``` I have a script that downloads files from Azure using Logstash to send them to ES. In the end there will be 90 000+ files which is the reason I have close\_older as an option. The problem I have is that I get a lot of error messages claiming JSON parse error. In the example I run, I have 8600 JSON objects in 1000 files. The number of documents that gets indexed in ES varies from 6500 - 8500. I am quite confident that the files contains properly formatted JSON objects. If there really was a problem with the formatting I would assume that I got the same number of errors whenever I ran a test with my sample files but I get different results every time. Does anyone have a clue what could cause this? Running ES/LS 6.3 with x-pack Best regards, Mattias ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 6:45am UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/2 "2018-06-29T06:45:53Z")
Can you share a file that gives different results from run to run? Are each JSON object in a file on a single line? ---
**Author:** ![Matjo](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@Matjo](https://discuss.elastic.co/u/Matjo)\ **Post date:** [June 29, 2018, 7:37am UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/3 "2018-06-29T07:37:17Z")
I will see if I can create a file that behaves in that way. The JSON objects are not on a single line, they are new line delimited. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 7:39am UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/4 "2018-06-29T07:39:47Z")
If they are not in a single line I believe you will need to use a multiline codec to make sure the full JSON object is in a single event. ---
**Author:** ![Matjo](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@Matjo](https://discuss.elastic.co/u/Matjo)\ **Post date:** [June 29, 2018, 7:46am UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/5 "2018-06-29T07:46:35Z")
Ok, how would the JSON objects be separated if they are in a single line? " ", "," or "\t", what would be a multiline codec to use? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 7:49am UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/6 "2018-06-29T07:49:21Z")
Each JSON object should be on a separate line. ---
**Author:** ![Matjo](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@Matjo](https://discuss.elastic.co/u/Matjo)\ **Post date:** [June 29, 2018, 1:55pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/7 "2018-06-29T13:55:10Z")
Thanks for taking time to answer my question! I think we may have misunderstood each other. Just to make sure I understand you correctly here is a more detailed explanation: The files look like this: ``` {JSON}\n (1) {JSON}\n (2) {JSON} (3) ``` I want to index this as three different documents in Elastic. 1. Do I need a multiline codec for this? (it is not one object on multiple rows) 2. If not, what could be potential errors that cause Parse errors? Best regards, Mattias ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 29, 2018, 2:07pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/8 "2018-06-29T14:07:22Z")
If each object is on a single line and followed by a newline you will not need to use the multiline codec. > [@Matjo](#): > > If not, what could be potential errors that cause Parse errors? I would recommend looking for parse failures in your config and write these to a file. Without knowing what they look like it is hard to speculate about what could be wrong. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 27, 2018, 2:07pm UTC](https://discuss.elastic.co/t/varying-amount-of-json-parse-errors-when-parsing-the-same-files/137864/9 "2018-07-27T14:07:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.