Getting _jsonparsefailure in logstash

Elastic Stack Logstash
1

Hi I am trying to parse a json file. I have tried troubleshooting with suggestions from stackoverflow (links at bottom)but none have worked for me. I am hoping someone has some insight on probably a silly mistake I am making.

I have tried using only the json codec, only the json filter, as well as both. For some reason I am still getting this _jsonparsefailure. What can I do to get this to work?

Thanks in advance!

My json file:

{
  "log": {
    "version": "1.2",
    "creator": {
      "name": "WebInspector",
      "version": "537.36"
    },
    "pages": [
      {
        "startedDateTime": "2015-10-13T20:28:46.081Z",
        "id": "page_1",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 377.8560000064317,
          "onLoad": 377.66200001351535
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:01.734Z",
        "id": "page_2",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 1444.0670000039972,
          "onLoad": 2279.20100002666
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:04.014Z",
        "id": "page_3",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 1802.0240000041667,
          "onLoad": 2242.4060000048485
        }
      },
      {
        "startedDateTime": "2015-10-13T20:29:09.224Z",
        "id": "page_4",
        "title": "https://demo.com",
        "pageTimings": {
          "onContentLoad": 274.82699998654425,
          "onLoad": 1453.034000005573
        }
      }
    ]
  }
}

My logstash conf:

input {
    file {
       type => "json"
       path => "/Users/anonymous/Documents/demo.json"
       start_position => beginning
    }
}

filter{
    json{
        source => "message"
   }
}


output { 
  elasticsearch { host => localhost protocol => "http" port => "9200" } 
  stdout { codec => rubydebug } 
}

Output I am getting from logstash hopefully with clues:

Trouble parsing json {:source=>"message", :raw=>"        \"startedDateTime\": \"2015-10-19T18:05:37.887Z\",", :exception=>#<TypeError: can't convert String into Hash>, :level=>:warn}
{
       "message" => "      {",
      "@version" => "1",
    "@timestamp" => "2015-10-26T20:05:53.096Z",
          "host" => "15mbp-09796.local",
          "path" => "/Users/anonymous/Documents/demo.json",
          "type" => "json",
          "tags" => [
        [0] "_jsonparsefailure"
    ]
}
1 Like
2

First of all the JSON document you provided is not valid JSON as there is a stray comma on the third line from the end. You may also need to use a multiline codec to assemble the document into the message field before using the son filter to parse it. Something like this should work, but it is possible that the multiline codec will not release the event until the next one is received.

 input {
    file {
       type => "json"
       path => "/Users/anonymous/Documents/demo.json"
       start_position => beginning
       codec => multiline {
           pattern => "^{"
           negate => "true"
           what => "previous"
       }
    }
}

filter{
    json{
        source => "message"
    }
}

output { 
  stdout { codec => rubydebug } 
}
3

Thanks, I removed the comma, I truncated the original file and missed the comma.

Unfortunately, this was the output I received. I looks like it was successful in putting the data into the message field, but the message field was not parsed into their respective fields.

Any thoughts?

Logstash output:

Trouble parsing json {:source=>"message", :raw=>"{\n  \"log\": {\n    \"version\": \"1.2\",\n    \"creator\": {\n      \"name\": \"WebInspector\",\n      \"version\": \"537.36\"\n    },\n    \"pages\": [\n      {\n        \"startedDateTime\": \"2015-10-19T18:05:37.887Z\",\n        \"id\": \"page_3\",\n        \"title\": \"https://demo.com/\",\n        \"pageTimings\": {\n