# Json parse failure even when the json is valid

**URL:** <https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609>\
**Category:** Logstash\
**Created:** [June 27, 2018, 12:05pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609 "2018-06-27T12:05:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pooja](https://avatars.discourse-cdn.com/v4/letter/p/a5b964/32.png) [@Pooja](https://discuss.elastic.co/u/Pooja)\
**Post date:** [June 27, 2018, 12:05pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/1 "2018-06-27T12:05:49Z")

</div>

conf filter:  
filter{  
mutate {  
gsub =\> [  
"message","\r","",  
"message","\n","",  
"message"," ",""  
]  
}  
grok {  
match =\> { "message" =\> "^%{YEAR:year}-%{MONTH:month}-%{MONTHDAY:day} %{TIME:time} %{GREEDYDATA:pam} [%{GREEDYDATA:data}] %{LOGLEVEL:security} %{GREEDYDATA:class} - %{WORD:jsn} = { %{GREEDYDATA:mth}}" }  
}  
json {  
source =\> "mth"  
target =\> "parsemethod"  
}  
if [jsn] != "Message" {  
drop { }  
}  
#if [method] != "getDeviceInfoResponse" {  
#drop { }  
#}  
}

Logstash parser:  
{  
"@timestamp" =\> 2018-06-27T11:51:13.948Z,  
"@version" =\> "1",  
"mth" =\> " "getDeviceInfoResponse" : { "messageId" : "123", "message" : "Success", "rudi" : "REF  
01234567891234-16", "geoRegion" : "North America", "country" : "US", "status" : "Registered", "connectivit  
y" : "To be connected", "sharedAccessKey" : "" }",  
"day" =\> "08",  
"month" =\> "May",  
"jsn" =\> "Message",  
"path" =\> "C:\Users\HOSDURGP\Documents\Varsha\Varsha\rsp-geodir-app.log",  
"year" =\> "2018",  
"class" =\> " com.roche.rsp.geodir.rabbitmq.Publisher",  
"tags" =\> [  
[0] "multiline",  
[1] "\_jsonparsefailure"  
],  
"data" =\> "SimpleAsyncTaskExecutor-1",  
"security" =\> "INFO",  
"time" =\> "15:37:24",  
"host" =\> "RDR-VM592",  
"pam" =\> "PM",  
"message" =\> "2018-May-08 15:37:24 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Publisher -  
Message = { "getDeviceInfoResponse" : { "messageId" : "123", "message" : "Success", "rudi" : "REF01234567  
891234-16", "geoRegion" : "North America", "country" : "US", "status" : "Registered", "connectivity" : "  
To be connected", "sharedAccessKey" : "" }}2018-May-08 15:38:31 PM Routing Key = geodir.msgbroker.cmdprocess"  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 27, 2018, 12:21pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/2 "2018-06-27T12:21:36Z")

</div>

The `mth` field is not valid JSON. It's missing a `{` at the beginning.

---

<div class="post-metadata">

**Author:** ![Pooja](https://avatars.discourse-cdn.com/v4/letter/p/a5b964/32.png) [@Pooja](https://discuss.elastic.co/u/Pooja)\
**Post date:** [June 27, 2018, 1:31pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/3 "2018-06-27T13:31:13Z")

</div>

Hey Magnus,

Thanks for the reply.  
But when I check the mth filed value on json lint, I found that it is a valid json.

"mth" =\> " "getDeviceInfoResponse" : { "messageId" : "123", "message" : "Success", "rudi" : "REF  
01234567891234-16", "geoRegion" : "North America", "country" : "US", "status" : "Registered", "connectivit  
y" : "To be connected", "sharedAccessKey" : "" }",

Do you think I should be adding a '{' at the beginning and append it with '}' at the end?

---

<div class="post-metadata">

**Author:** ![Pooja](https://avatars.discourse-cdn.com/v4/letter/p/a5b964/32.png) [@Pooja](https://discuss.elastic.co/u/Pooja)\
**Post date:** [June 27, 2018, 1:35pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/4 "2018-06-27T13:35:52Z")

</div>

FYI

Trying to parse the following lines on the log file:  
2018-May-08 15:05:34 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Subscriber - RECEIVED REQUEST : storeDeviceInfo  
2018-May-08 15:05:34 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Publisher - Message = {  
"storeEvent" : {  
"eventType" : "DEVICE UPDATE",  
"eventName" : "Device registration",  
"eventMessage" : "New Device registered, RUDI :REF01234567891234-12",  
"timestamp" : "Tue May 08 15:05:34 CEST 2018"  
}  
}  
2018-May-08 15:38:31 PM Routing Key = geodir.msgbroker.eventstore  
2018-May-08 15:05:34 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.service.GeoDirectoryService - EVENT GENERATED.  
2018-May-08 15:05:34 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.service.GeoDirectoryService - DEVICE INFO SAVED, RUDI :REF01234567891234-12  
2018-May-08 15:05:34 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Publisher - Message = {  
"storeDeviceInfoResponse" : {  
"messageId" : "123",  
"rudi" : "REF01234567891234-12",  
"message" : "SAVED"  
}  
}

Is it possible for you to help me out with the correct filter to parse the json out of this log file?

Thanks

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 27, 2018, 2:02pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/5 "2018-06-27T14:02:52Z")

</div>

If you format your logs as preformatted text using markdown notation or the `</>` toolbar button we can see what it actually looks like.

Wait, this part of your grok expression is wrong:

```
... { %{GREEDYDATA:mth}}

```

Change it to

```
... %{GREEDYDATA:mth}

```

so that the braces are included in the `mth` field.

---

<div class="post-metadata">

**Author:** ![Pooja](https://avatars.discourse-cdn.com/v4/letter/p/a5b964/32.png) [@Pooja](https://discuss.elastic.co/u/Pooja)\
**Post date:** [June 27, 2018, 2:23pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/6 "2018-06-27T14:23:59Z")

</div>

The reason why I used that grok expression { %{GREEDYDATA:mth}} is because it reads the next two lines along as they don't match the pattern provided in the multiline codec in the input plugin parameters.

Here is how it parses if the grok expression %{GREEDYDATA:mth} is used.  
{  
"host" =\> "RDR-VM592",  
"jsn" =\> "Message",  
"year" =\> "2018",  
"month" =\> "May",  
"time" =\> "15:38:21",  
"class" =\> " com.roche.rsp.geodir.rabbitmq.Publisher",  
"@timestamp" =\> 2018-06-27T14:19:50.520Z,  
"pam" =\> "PM",  
"mth" =\> "{ "updateSasKeyResponse" : { "messageId" : "123", "rudi" : "REF01234567891234-16", "mes  
sage" : "UPDATED SAS KEY" }}2018-May-08 15:38:31 PM Routing Key = geodir.msgbroker.cmdprocess2018-May-08 15:38:31 PM  
[SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Subscriber - RECEIVED REQUEST : updateSasKey2018-May-08  
15:38:31 PM [SimpleAsyncTaskExecutor-1] ERROR com.roche.rsp.geodir.service.GeoDirectoryService - FAILED TO UPDATE SAS K  
EY, RUDI :REF01234567891234-16",  
"message" =\> "2018-May-08 15:38:21 PM [SimpleAsyncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Publisher -  
Message = { "updateSasKeyResponse" : { "messageId" : "123", "rudi" : "REF01234567891234-16", "message" :   
"UPDATED SAS KEY" }}2018-May-08 15:38:31 PM Routing Key = geodir.msgbroker.cmdprocess2018-May-08 15:38:31 PM [SimpleAs  
yncTaskExecutor-1] INFO com.roche.rsp.geodir.rabbitmq.Subscriber - RECEIVED REQUEST : updateSasKey2018-May-08 15:38:31  
PM [SimpleAsyncTaskExecutor-1] ERROR com.roche.rsp.geodir.service.GeoDirectoryService - FAILED TO UPDATE SAS KEY, RUDI :  
REF01234567891234-16",  
"@version" =\> "1",  
"tags" =\> [  
[0] "multiline",  
[1] "\_jsonparsefailure"  
],  
"data" =\> "SimpleAsyncTaskExecutor-1",  
"path" =\> "C:\Users\xxxx\xxxx\xxxxx\xxxxxx\rsp-geodir-app.log",  
"security" =\> "INFO",  
"day" =\> "08"  
}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 27, 2018, 4:27pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/7 "2018-06-27T16:27:17Z")

</div>

Why is your multiline config joining multiline lines into the same Logstash event? That seems broken.

**I repeat:** If you format your logs as preformatted text using markdown notation or the `</>` toolbar button we can see what they actually look like.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2018, 4:27pm UTC](https://discuss.elastic.co/t/json-parse-failure-even-when-the-json-is-valid/137609/8 "2018-07-25T16:27:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.