# Error parsing JSON content from Http

**URL:** https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069
**Category:** Logstash
**Created:** [July 11, 2019, 4:24pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069 "2019-07-11T16:24:47Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![SamDi](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@SamDi](https://discuss.elastic.co/u/SamDi)
#### Post date: [July 11, 2019, 4:24pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/1 "2019-07-11T16:24:47Z")

</div>

Hello to everybody.  
I am really noob on elastic stack and I am trying to import some json information with the HTTP input to Elasticsarch, but I recive a parse error:  
`exception=>#<LogStash::Json::ParserError: Unrecognized token 'json': was expecting ('true', 'false' or 'null')`

This is how i recive the information on my elasticsearch:

```
"_type" : "_doc",
        "_id" : "vCS44WsBBF5AknIKMFMZ",
        "_score" : 1.0,
        "_source" : {
          "message" : """json={"id":"136030541","domain":"sdl","priority":"5.0","context":"my.alert.sdl.Error500","srcIp":"","dstIp":"","srcPort":"","dstPort":"","category":"my.context","protocol":"","username":"","application":"","status":"4","creation_date":"Thu Jul 11 15:47:52 UTC 2019","ack_status_date":null,"digest":"e738f645464be99ff223b504ec4a213d92ce8c6f","scaled":"false","srcHost":"","dstHost":"","extraData":"{\"clientIpAddress\":\"null\",\"timeTaken\":\"463\",\"protocol\":\"HTTP+1.1\",\"bytesTransferred\":\"677\",\"method\":\"POST\",\"cookie\":\"gaqfse5dpcm690jdh5ho1f00o2%3A-\",\"userAgent\":\"Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+NT+5.1%29\",\"referralUri\":\"http%3A%2F%2Fwww.logcasts.com%2Fcart.do%3Faction%3Dview%26itemId%3DLOG-56%26product_id%3D235-40ER0-J3308%26JSESSIONID%3DSD3SL2FF4ADFF4\",\"uri\":\"%2Fcategory.screen%3Fcategory_id%3DFURNITURE%26JSESSIONID%3DSD3SL2FF4ADFF4\",\"eventdate\":\"2019-07-11+11%3A56%3A57.04\",\"timestamp\":\"11%2FJul%2F2019%3A11%3A56%3A53+%2B0000\",\"statusCode\":\"500\"}","engine":"pilot.my.alert.samps.Error500","textMessage":"Error 500","shortMessage":"Error 500","alertname":"Error500","categoryname":"My Alerts","subcategoryname":"Personal"}""",
          "headers" : {
            "request_path" : "/",
            "request_method" : "POST",
            "http_accept" : null,
            "content_length" : "1219",
            "connection" : "Keep-Alive",
            "http_user_agent" : "Apache-HttpClient/4.5.2 (Java/1.8.0_191)",
            "http_host" : "XXXXXXXXXXXXXXX:8080",
            "http_version" : "HTTP/1.1",
            "content_type" : "application/x-www-form-urlencoded"
          },
          "host" : "XXXXXXXXXXXXX",
          "@version" : "1",
          "@timestamp" : "2019-07-11T15:47:52.855Z"
        }

```

I want to parse the message field with this configuration:

```
filter {
  json {
    source => "message"
 }
}

```

Could anyone help me with this? I think that could be the format of the message field, but i dont know how i could alter it.

Thanks to all.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 11, 2019, 4:55pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/2 "2019-07-11T16:55:44Z")

</div>

Yes, the problem is that the message field has other stuff prefixed the JSON. Please edit your post, select the configuration, and click on \</\> in the toolbar above the edit pane. If you look in the preview pane on the right it will start appearing

```
like this

```

Then do the same for document data. At the moment the browser is removing all the \ characters that are escaped in the message field, so it is too hard to tell what you need to remove.

---

<div class="post-metadata">

### Author: ![SamDi](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@SamDi](https://discuss.elastic.co/u/SamDi)
#### Post date: [July 11, 2019, 7:17pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/3 "2019-07-11T19:17:30Z")

</div>

Thanks Badger, post edited.  
I must to add that i cant change the way i recibe the data, I have to edit the message content with a logstash filter, I suppose.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 11, 2019, 9:39pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/4 "2019-07-11T21:39:38Z")

</div>

Something like

```
    mutate { gsub => ["message", '^""json=', "", "message", '""$', ""] }
    json { source => "message" target => "jsondata1" }
    json { source => "[jsondata1][extraData]" target => "jsondata2" }
```

---

<div class="post-metadata">

### Author: ![SamDi](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@SamDi](https://discuss.elastic.co/u/SamDi)
#### Post date: [July 12, 2019, 7:05am UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/5 "2019-07-12T07:05:19Z")

</div>

Thanks Badger,  
I try it but dont work. This is the console output.

```
[WARN] 2019-07-12 06:51:30.307 [[main]>worker1] json - Error parsing json {:source=>"message", :raw=>"json={\"id\":\"136788731\",\"domain\":\"spb\",\"priority\":\"5.0\",\"context\":\"my.alert.spb.Error500\",\"srcIp\":\"\",\"dstIp\":\"\",\"srcPort\":\"\",\"dstPort\":\"\",\"category\":\"my.context\",\"protocol\":\"\",\"username\":\"\",\"application\":\"\",\"status\":\"4\",\"creation_date\":\"Fri Jul 12 06:51:30 UTC 2019\",\"ack_status_date\":null,\"digest\":\"48b7c6a38d3566dbfa9f4580104056a0351f00ff\",\"scaled\":\"false\",\"srcHost\":\"\",\"dstHost\":\"\",\"extraData\":\"{\\\"clientIpAddress\\\":\\\"null\\\",\\\"timeTaken\\\":\\\"227\\\",\\\"protocol\\\":\\\"HTTP+1.1\\\",\\\"bytesTransferred\\\":\\\"3317\\\",\\\"method\\\":\\\"GET\\\",\\\"cookie\\\":\\\"3djv1l0ebi7cmsai1131pf2a65%3A-\\\",\\\"userAgent\\\":\\\"Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+NT+5.1%29\\\",\\\"referralUri\\\":\\\"http%3A%2F%2Fwww.google.com%2Fproduct.screen%3Fproduct_id%3DL98-72BOK-SKD00%26JSESSIONID%3DSD3SL9FF8ADFF9\\\",\\\"uri\\\":\\\"%2Fcategory.screen%3Fcategory_id%3DFURNITURE%26JSESSIONID%3DSD3SL9FF8ADFF9\\\",\\\"eventdate\\\":\\\"2019-07-12+06%3A45%3A18.141\\\",\\\"timestamp\\\":\\\"12%2FJul%2F2019%3A06%3A44%3A56+%2B0000\\\",\\\"statusCode\\\":\\\"500\\\"}\",\"engine\":\"pilot.my.alert.spb.Error500\",\"textMessage\":\"Error 500\",\"shortMessage\":\"Error 500\",\"alertname\":\"Error500\",\"categoryname\":\"My Alerts\",\"subcategoryname\":\"Personal\"}", :exception=>#<LogStash::Json::ParserError: Unrecognized token 'json': was expecting ('true', 'false' or 'null')
 at [Source: (byte[])"json={"id":"136788731","domain":"spb","priority":"5.0","context":"my.alert.spb.Error500","srcIp":"","dstIp":"","srcPort":"","dstPort":"","category":"my.context","protocol":"","username":"","application":"","status":"4","creation_date":"Fri Jul 12 06:51:30 UTC 2019","ack_status_date":null,"digest":"48b7c6a38d3566dbfa9f4580104056a0351f00ff","scaled":"false","srcHost":"","dstHost":"","extraData":"{\"clientIpAddress\":\"null\",\"timeTaken\":\"227\",\"protocol\":\"HTTP+1.1\",\"byt"[truncated 691 bytes]; line: 1, column: 6]>}

```

and this is how i recive on elasticsearch:

```
# _score - 
  	t _type	_doc
   	t headers.connection	Keep-Alive
   	t headers.content_length	1187
 	? headers.content_type application/x-www-form-urlencoded
   	t headers.http_accept - 
   	t headers.http_host	XXXXXXXXXXXXXXX
   	t headers.http_user_agent	Apache-HttpClient/4.5.2 (Java/1.8.0_191)
   	t headers.http_version	HTTP/1.1
   	t headers.request_method	POST
   	t headers.request_path	/
   	t host	XXXXXXXXXXXXXXXXX
   	t message	json={"id":"136788651","domain":"spb","priority":"5.0","context":"my.alert.spb.Error500","srcIp":"","dstIp":"","srcPort":"","dstPort":"","category":"my.context","protocol":"","username":"","application":"","status":"4","creation_date":"Fri Jul 12 06:49:35 UTC 2019","ack_status_date":null,"digest":"0c86affb0499bac0f1413ab2a480c5fe93a605a7","scaled":"false","srcHost":"","dstHost":"","extraData":"{\"clientIpAddress\":\"%2F70.44.164.7\",\"timeTaken\":\"370\",\"protocol\":\"HTTP+1.1\",\"bytesTransferred\":\"1066\",\"method\":\"GET\",\"cookie\":\"3djv1l0ebi7cmsai1131pf2a65%3A-\",\"userAgent\":\"Opera%2F9.01+%28Windows+NT+5.1%3B+U%3B+en%29\",\"referralUri\":\"http%3A%2F%2Fwww.logcasts.com%2Foldlink%3Fitem_id%3DLOG-77%26port%3D161%26JSESSIONID%3DSD8SL10FF6ADFF8\",\"uri\":\"%2Fproduct.screen%3Fproduct_id%3DL98-72BOK-SKD00%26JSESSIONID%3DSD8SL10FF6ADFF8\",\"eventdate\":\"2019-07-12+06%3A44%3A06.039\",\"timestamp\":\"12%2FJul%2F2019%3A06%3A43%3A44+%2B0000\",\"statusCode\":\"500\"}","engine":"pilot.my.alert.spb.Error500","textMessage":"Error 500","shortMessage":"Error 500","alertname":"Error500","categoryname":"My Alerts","subcategoryname":"Personal"}
 	? tags _jsonparsefailure

```

I am reading documentation about mutate filter and gsub... Thanks again.

I think that i only need to remove de first "json=" text and all`"\"`. This way i have a valid json object but with anoter json object inside, "ExtraData", witch dont parse correctly because if i remove all the "" the field look like this:

`"extraData":"{"clientIpAddress":"null","timeTaken":"227","protocol":"HTTP+1.1","bytesTransferred":"3317","method":"GET","cookie":"3djv1l0ebi7cmsai1131pf2a65%3A-","userAgent":"Mozilla%2F4.0+%28compatible%3B+MSIE+6.0%3B+Windows+NT+5.1%29","referralUri":"http%3A%2F%2Fwww.google.com%2Fproduct.screen%3Fproduct_id%3DL98-72BOK-SKD00%26JSESSIONID%3DSD3SL9FF8ADFF9","uri":"%2Fcategory.screen%3Fcategory_id%3DFURNITURE%26JSESSIONID%3DSD3SL9FF8ADFF9","eventdate":"2019-07-12+06%3A45%3A18.141","timestamp":"12%2FJul%2F2019%3A06%3A44%3A56+%2B0000","statusCode":"500"}"`

Any idea to a good mutate filter? I still working on it.

Thanks.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 12, 2019, 4:48pm UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/6 "2019-07-12T16:48:55Z")

</div>

```
mutate { gsub => ["message", '^json=', ""] }

```

That will remove the json= at the beginning.

You do not need to remove the \ in ExtraData, it is just the way logstash displays double quotes within a double quoted string.

---

<div class="post-metadata">

### Author: ![SamDi](https://avatars.discourse-cdn.com/v4/letter/s/e9bcb4/32.png) [@SamDi](https://discuss.elastic.co/u/SamDi)
#### Post date: [July 15, 2019, 10:14am UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/7 "2019-07-15T10:14:08Z")

</div>

Really thanks you Badger, it was as easy as remove "json="

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 12, 2019, 10:14am UTC](https://discuss.elastic.co/t/error-parsing-json-content-from-http/190069/8 "2019-08-12T10:14:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.