Why _jsonparsefailure?

Elastic Stack Logstash
1

Having _jsonparsefailure, and json parser says messsage is good json.

I have this configuration file:

input {
http_poller {
urls => {
url => "https://urlhost:443/remote/core.executeQuery?queryId=###&:output=json"
}
cacert => "/etc/logstash/conf.d/ssl/trust.pem"
truststore => "/etc/logstash/conf.d/ssl/trust.jks"
user => "user"
password => "xxxxxx"
truststore_password => "xxxxxxx"
schedule => { cron => "*/1 * * * *"}
codec => "json"
}
}

filter {
mutate {
gsub => [
"message", "^OK:\r\n", ""
]
}
json {
source => "message"
}
}

output{
elasticsearch {
manage_template => false
hosts => "localhost:9200"
index => "indexepo-%{+YYYY.MM.dd}"
}
stdout{}
}

Message:
OK:
[ { "EPOLeafNode.LastUpdate" : null, "EPOLeafNode.NodeName" : "x.x.x.x", "EPOComputerProperties.IPV6" : null, "EPOComputerProperties.IsPortable" : -1, "EPOComputerProperties.OSVersion" : "", "EPOComputerProperties.UserName" : "" }, { "EPOLeafNode.LastUpdate" : null, "EPOLeafNode.NodeName" : "x.x.x.x", "EPOComputerProperties.IPV6" : null, "EPOComputerProperties.IsPortable" : -1, "EPOComputerProperties.OSVersion" : "", "EPOComputerProperties.UserName" : "" }, { "EPOLeafNode.LastUpdate" : null, "EPOLeafNode.NodeName" : "pc1", "EPOComputerProperties.IPV6" : null, "EPOComputerProperties.IsPortable" : -1, "EPOComputerProperties.OSVersion" : "", "EPOComputerProperties.UserName" : "" }, { "EPOLeafNode.LastUpdate" : "2019-04-03T07:55:31-04:00", "EPOLeafNode.NodeName" : "pc2", "EPOComputerProperties.IPV6" : "0:0:0:0:0:xxx:xxxx", "EPOComputerProperties.IsPortable" : 0, "EPOComputerProperties.OSVersion" : "6.1", "EPOComputerProperties.UserName" : "User3" }, { "EPOLeafNode.LastUpdate" : "2019-04-03T07:55:31-04:00", "EPOLeafNode.NodeName" : "PC5", "EPOComputerProperties.IPV6" : "0:0:0:0:0:Fxxxxxx", "EPOComputerProperties.IsPortable" : 0, "EPOComputerProperties.OSVersion" : "6.1", "EPOComputerProperties.UserName" : "User7" } ]

Kibana discovery Error:
April 4th 2019, 11:43:03.522
@timestamp: April 4th 2019, 11:43:03.522
tags: _jsonparsefailure
@version:1
message: The above message

2

What field do you expect the array to show up in? You need to give the json filter a target option.

Another approach, worse in every way, would be

mutate { gsub => [ "message", "$", " }", "message", "^", '{ "someName": ' ] }
3

Added the target field and work, but still not getting the fields, just the entire message in the new tartget field.

4 
json { source => "message" target => "someField" }

gets me

 "someField" => [
    [0] {
        "EPOComputerProperties.IsPortable" => -1,
              "EPOComputerProperties.IPV6" => nil,
                    "EPOLeafNode.NodeName" => "x.x.x.x",
         "EPOComputerProperties.OSVersion" => "",
                  "EPOLeafNode.LastUpdate" => nil,
          "EPOComputerProperties.UserName" => ""
    },
    [1] { ...

Is that not what you get?

5

Yes!!! Right.

But all in the same "someField"

I need all the values separate.

6

Screenshot_1
Screenshot_1.png766×804 37.3 KB

7

Can you be more explicit about what you want? Do you want each entry in the array to be a separate event?

8

The whole event is a list in json. Each item on the list is a PC from where i work.

Each pc needs to be in a different row.

Right now they are all in the "evento" field.

Is there anyway to achieve this?

9

If you are OK with the fields being nested you could use

json { source => "message" target => "someField" remove_field => "message" }
split { field => "someField" }

If you want them at the top level then

json { source => "message" target => "[@metadata][someField]" remove_field => "message" }
split { field => "[@metadata][someField]" }
ruby { code => 'event.get("[@metadata][someField]").each { |k, v| event.set(k, v) }' }
10

Thanks a lot! Is working.

Taking some time to index as there are more than 10k values.

Thanks again.