Split Filter Parse Failure

Elastic Stack Logstash
1

Hi, please help with splitting the following JSON at "value":

{"count":3,
"value":[
{"id":232, "buildNumber":"20180706.7", "status":"completed"},
{"id":231, "buildNumber":"20180706.6", "status":"completed"},
{"id":229, "buildNumber":"20180706.4", "status":"completed"}
]}

My Logstash Config File:

input {
file {
path => "C:\Users\Rahul J\Downloads\compressed.json"
codec => "plain"
sincedb_path => "/dev/null"
start_position => "beginning"
}

}

filter {
split {field => "[value]"}

mutate {
add_field => {
"id" => "This is new ID"
"buildNumber" => "This was an experiment"
"status" => "This was an experiment"
}
}
}

output {
elasticsearch {
action => "index"
hosts => "localhost:9200"
index => "rahul"
workers => 1
}
stdout {
codec => rubydebug
}
}

Logstash Output:
[2018-08-24T12:54:43,997][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
[2018-08-24T12:54:43,997][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
[2018-08-24T12:54:43,998][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
[2018-08-24T12:54:44,002][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
[2018-08-24T12:54:44,004][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
[2018-08-24T12:54:44,005][WARN ][logstash.filters.split ] Only String and Array types are splittable. field:[value] is of type = NilClass
{
"@timestamp" => 2018-08-24T17:54:43.842Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => "{"count":3,\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}
{
"@timestamp" => 2018-08-24T17:54:43.879Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => "\t{"id":229, "buildNumber":"20180706.4", "status":"completed"}\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}
{
"@timestamp" => 2018-08-24T17:54:43.879Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => "\t{"id":231, "buildNumber":"20180706.6", "status":"completed"},\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}
{
"@timestamp" => 2018-08-24T17:54:43.877Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => ""value":[\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}
{
"@timestamp" => 2018-08-24T17:54:43.879Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => "]}\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}
{
"@timestamp" => 2018-08-24T17:54:43.878Z,
"path" => "C:\Users\Rahul J\Downloads\compressed.json",
"buildNumber" => "This was an experiment",
"host" => "DESKTOP-9LJJQ4J",
"message" => "\t{"id":232, "buildNumber":"20180706.7", "status":"completed"},\r",
"tags" => [
[0] "_split_type_failure"
],
"@version" => "1",
"status" => "This was an experiment",
"id" => "This is new ID"
}

I want to split the 'value' field that is a JSON array into multiple JSONs and then parse these individual JSONs to extract the value of keys: ID, buildNumber, status. The outcome will be appended to the event containing explicit id, buildNumber and status fields created by the Mutate filter.

Please guide. Thank you in advance.

2

The split filter fails because none of the events you've listed contains a value field. To slurp a multiline JSON file into a single event you need to use a multiline codec (and set its auto_flush_interval option to e.g. 5).

sincedb_path => "/dev/null"

On Windows use "nul", not "/dev/null".

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.