# Error with parsing Azure json multiline files

**URL:** <https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307>\
**Category:** Logstash\
**Created:** [March 3, 2017, 11:58am UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307 "2017-03-03T11:58:50Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![111148](https://avatars.discourse-cdn.com/v4/letter/1/eb8c5e/32.png) [@111148](https://discuss.elastic.co/u/111148)\
**Post date:** [March 3, 2017, 11:58am UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/1 "2017-03-03T11:58:50Z")

</div>

Hello,  
Please advise how to solve this.

 ![](https://us1.discourse-cdn.com/elastic/original/3X/b/2/b2da3be41a779cbe27a03080c17bb47e15926768.png)  
Here is my logstash config file:  
 ![](https://us1.discourse-cdn.com/elastic/original/3X/0/4/041052eb5b1f6fe3a59fe9a040f49f972d9c0e55.PNG)

---

<div class="post-metadata">

**Author:** ![111148](https://avatars.discourse-cdn.com/v4/letter/1/eb8c5e/32.png) [@111148](https://discuss.elastic.co/u/111148)\
**Post date:** [March 3, 2017, 12:49pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/2 "2017-03-03T12:49:50Z")

</div>

Added this part to logstash config.

![](https://us1.discourse-cdn.com/elastic/original/3X/c/f/cf49cd3c7d22763fd151511817f3d876fafe82e5.png)

But still no success. Logstash can not parse json.  
any ideas?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 3, 2017, 1:01pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/3 "2017-03-03T13:01:23Z")

</div>

Please don't paste images of text as they can be very hard to read. Start troubleshooting this by removing the elasticsearch output and instead use the stdout plugin with a rubydebug codec, so you can see what the events Logstash processes look like.

---

<div class="post-metadata">

**Author:** ![111148](https://avatars.discourse-cdn.com/v4/letter/1/eb8c5e/32.png) [@111148](https://discuss.elastic.co/u/111148)\
**Post date:** [March 3, 2017, 1:03pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/4 "2017-03-03T13:03:20Z")

</div>

Ok, thanks for your advice. but i do not know how to use Rubydebug.  
Should i install ruby for this? i am not a programmer but only devops.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 3, 2017, 1:25pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/5 "2017-03-03T13:25:15Z")

</div>

Create the following output block:

```
output {
  stdout {
    codec => rubydebug
  }
}

```

This will log messages to stdout so that you can inspect them.

---

<div class="post-metadata">

**Author:** ![111148](https://avatars.discourse-cdn.com/v4/letter/1/eb8c5e/32.png) [@111148](https://discuss.elastic.co/u/111148)\
**Post date:** [March 3, 2017, 1:35pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/6 "2017-03-03T13:35:30Z")

</div>

Added but i can not see any messages in powershell console after logstash has been started. (and in logstash files as well)

[2017-03-03T13:33:25,634][INFO][logstash.inputs.azureblob] Using version 0.9.x input plugin 'azureblob'. This plugin should work but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin.  
[2017-03-03T13:33:26,068][INFO][logstash.pipeline] Starting pipeline {"id"=\>"main", "pipeline.workers"=\>2, "pipeline.batch.size"=\>125, "pipeline.batch.delay"=\>5, "pipeline.max\_inflight"=\>250}  
[2017-03-03T13:33:26,099][INFO][logstash.pipeline] Pipeline main started  
[2017-03-03T13:33:26,240][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}

---

<div class="post-metadata">

**Author:** ![111148](https://avatars.discourse-cdn.com/v4/letter/1/eb8c5e/32.png) [@111148](https://discuss.elastic.co/u/111148)\
**Post date:** [March 3, 2017, 2:13pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/7 "2017-03-03T14:13:00Z")

</div>

Managed to see an error only after deleting existing index (and reindexing existing data).

at [Source: [B@7f4b9de9; line: 1, column: 16]\>}  
[2017-03-03T14:05:55,888][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"\t\t\t "level": "Error",\r", :exception=\>#\<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array,  
ject, 'true', 'false' or 'null')  
at [Source: [B@5b9388c5; line: 1, column: 13]\>}  
[2017-03-03T14:05:55,888][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"\t\t\t "location": "global",\r", :exception=\>#\<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, arr  
, object, 'true', 'false' or 'null')  
at [Source: [B@7aaacd26; line: 1, column: 16]\>}  
[2017-03-03T14:05:55,888][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"\t\t\t "properties": {"statusCode":"BadRequest","serviceRequestId":null,"statusMessage":"{\"error\":{\"code\":\"InvalidTemplate\",\"mess  
e\":\"Deployment template validation failed: 'The template reference 'PublicIP-LB-FE-0' is not valid: could not find template resource or resource copy with this name. Please see [https://aka.ms/arm-template-expressions/#reference](https://aka.ms/arm-template-expressions/#reference) for usage details.'.\"}}"}\r",  
xception=\>#\<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')  
at [Source: [B@3fc7bc5e; line: 1, column: 18]\>}  
[2017-03-03T14:05:55,888][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"\t\t}\r", :exception=\>#\<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for ROOT starting at [Source: [B@3e5a2208; line: 1, column: 0])  
at [Source: [B@3e5a2208; line: 1, column: 4]\>}  
[2017-03-03T14:05:55,904][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"\t]\r", :exception=\>#\<LogStash::Json::ParserError: Unexpected close marker ']': expected '}' (for ROOT starting at [Source: [B@1fbf7190; line: 1, column: 0])  
at [Source: [B@1fbf7190; line: 1, column: 3]\>}  
[2017-03-03T14:05:55,919][WARN][logstash.filters.json] Error parsing json {:source=\>"message", :raw=\>"}\r", :exception=\>#\<LogStash::Json::ParserError: Unexpected close marker '}': expected ']' (for ROOT starting at [Source: [B@6f43f4e1; line: 1, column: 0])  
at [Source: [B@6f43f4e1; line: 1, column: 2]\>}  
{  
"@timestamp" =\> 2017-03-03T14:05:55.466Z,  
"@version" =\> "1",  
"message" =\> "{\r",  
"tags" =\> [  
[0] "\_jsonparsefailure"  
]  
}  
{  
"@timestamp" =\> 2017-03-03T14:05:55.466Z,  
"@version" =\> "1",  
"message" =\> "\t"records": \r",  
"tags" =\> [  
[0] "\_jsonparsefailure"  
]  
}  
{  
"@timestamp" =\> 2017-03-03T14:05:55.466Z,  
"@version" =\> "1",  
"message" =\> "\t[\r",  
"tags" =\> [  
[0] "\_jsonparsefailure"  
]  
}  
{  
"@timestamp" =\> 2017-03-03T14:05:55.466Z,  
"@version" =\> "1",  
"message" =\> "\t\t{\r",  
"tags" =\> [  
[0] "\_jsonparsefailure"  
]  
}

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 3, 2017, 2:20pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/8 "2017-03-03T14:20:32Z")

</div>

It looks like your JSON documents pan multiple lines, so unless you can change the format of the files and get one JSOn object per line you will probably need to use a [multiline codec](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html) in order to assemble the JSON objects before parsing then with a [json filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-json.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 31, 2017, 2:26pm UTC](https://discuss.elastic.co/t/error-with-parsing-azure-json-multiline-files/77307/10 "2017-03-31T14:26:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.