Logstash config - JSON filter results with "_jsonparsefailure"

Elastic Stack Logstash
1

Hi,

My log file is in JSON format.
An example of a log line:

{"@timestamp":"2018-09-18T10:36:27.135+03:00","@version":1,"message":{"apiMethodType": "GET","elapsedTime": 1136},"caller_line_number":94}

I'm using Filebeat with the following prospectors settings:

filebeat.prospectors:

  • paths:
    • C:\Users\abc\Desktop\E.L.K\Log*.json
      input_type: log
      json.keys_under_root: true
      json.add_error_key: true

I'd like to get the values of the two fields withing the message: apiMethodType, elapsedTime.

My logstash config file is:
input {
beats {
port => "5044"
}
}

filter{
json{
source => "message"
target => "JsonMessage"
}
mutate {
add_field => {
"apiMethodType" => "%{[JsonMessage][apiMethodType]}"
"elapsedTime" => "%{[JsonMessage][elapsedTime]}"
}
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
index => ["logs-%{+YYYY.MM.dd}"]
}
}

The result looks as follows:

  {
    "_index" : "logs-2018.09.20",
    "_type" : "doc",
    "_id" : "hW2K9mUB55jPhmkgUZyh",
    "_score" : 1.0,
    "_source" : {
      "caller_line_number" : 94,
      "source" : "C:\\Users\\abc\\Desktop\\E.L.K\\Log\\Sample.json",
      "message" : "{\"apiMethodType\"=>\"GET\", \"elapsedTime\"=>1136}",
      "beat" : {
        "version" : "6.4.0",
        "name" : "IRA",
        "hostname" : "IRA"
      },
      "@timestamp" : "2018-09-20T10:28:02.342Z",
      "@version" : 1,
      "host" : {
        "name" : "IRA"
      },
      "apiMethodType" : "%{[JsonMessage][apiMethodType]}",
      "elapsedTime" : "%{[JsonMessage][elapsedTime]}",
      "offset" : 508,
      "tags" : [
        "beats_input_codec_plain_applied",
        "_jsonparsefailure"
      ]
    }
  }

Could you, please, assist me with understanding why am I getting the "_jsonparsefailure"?
What should I do in order to fetch the values of apiMethodType and elapsedTime correctly?

Thanks.

2

{"apiMethodType"=>"GET", "elapsedTime"=>1136} isn't a valid JSON string. Given the log example at the top and the configuration you've given it's very hard to understand how that ended up in your message field.

3

Even when I'm removing the filter the result looks like this:

{
"_index" : "logs-2018.09.20",
"_type" : "doc",
"_id" : "lG1J92UB55jPhmkgBpyJ",
"_score" : 1.0,
"_source" : {
"@timestamp" : "2018-09-20T14:01:18.541Z",
"caller_line_number" : 94,
"host" : {
"name" : "IRA"
},
"beat" : {
"name" : "IRA",
"hostname" : "IRA",
"version" : "6.4.0"
},
"source" : "C:\Users\abc\Desktop\E.L.K\Log\Sample.json",
"message" : "{"apiMethodType"=>"GET", "elapsedTime"=>1136}",
"tags" : [
"beats_input_codec_plain_applied"
],
"offset" : 0,
"@version" : 1
}
}
What am I missing?

Thanks.

4

What does the first line of Sample.json look like?

5

The log line looks like this:

{"@timestamp":"2018-09-18T10:36:27.135+03:00","@version":1,"message":{"apiMethodType": "GET","elapsedTime": 1136},"caller_line_number":94}

6

Okay. Then I have no idea how that line could possibly end up as the message field in a previous post.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.