JSON file input to logstash not working


#1

I am using json input to logstash and my output is elastic search.

Below is my input json file.

[
{
color: "red",
value: "#f00"
},
{
color: "green",
value: "#0f0"
},
{
color: "blue",
value: "#00f"
},
{
color: "cyan",
value: "#0ff"
},
{
color: "magenta",
value: "#f0f"
},
{
color: "yellow",
value: "#ff0"
},
{
color: "black",
value: "#000"
}
]

Below is my config file for logstash

input
{
file
{
path => "D:/ELK5.0/logstash-5.0.0/bin/jsontest.json"
sincedb_path => "D:/ELK5.0/logstash-5.0.0/bin/sincedb_path.txt"
start_position => "beginning"
}
}

filter {

}

output {
stdout { codec => json}
elasticsearch {
action => "index"
hosts => "127.0.0.1:9200"
index => "a1"
workers => 1
}
}

When i am using above config file i am getting output in kibana as each line of json file as one record.
I searched in elastic support they suggested to use multine.
I used below as my input but its not creating any index in elastic search.

input
{
file
{
codec => multiline
{
pattern => '^{'
negate => true
what => previous
}
path => "D:/ELK5.0/logstash-5.0.0/bin/jsontest.json"
sincedb_path => "D:/ELK5.0/logstash-5.0.0/bin/sincedb_path.txt"
start_position => "beginning"
}
}

MY expected results is 
> color                   value
> red		               #f00
> green                    #0f0
> blue	                   #00f
> cyan	                   #0ff
> magenta	               #f0f
> yellow	               #ff0
> black	                   #000

Please help me how can i get.

Thanks in advance


(Guy Boertje) #2

Firstly, the structure of the file you quoted is not valid JSON as encoded. It might be valid inside a javascript .js file but as a standalone pretty printed JSON array is it not. The fields of the inner objects MUST be quoted.

If you correct that and use the multiline codec, what output do you get when you use stdout + rubydebug as your output?

I suspect the first event message text will be

'[{\n"color": “red”,\n"value": “#f00”\n}'

As is, this will not parse as JSON because of the leading [ without a trailing ], you need to remove this [.

All other lines except the last you need to remove the trailing , and for the last line the trailing ].

Maybe:

  mutate {
    gsub => [
      # remove all leading '[' and trailing ',' or ']'
      "message", "(?:\\A\\[|[,\\]]\\z)", ""
    ]
  }

then you can use the json filter.

Check with stdout + rubydebug codec to verify.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.