Logstash sending json written by Java

Elastic Stack Logstash
1

I currently have code that writes a json array to a file. However, logstash only seems to attempt to parse it if I manually go in and touch the file myself, rather than take it in after the initial writing to the file. How could I change this behavior so that the json array is sent to logstash as soon as the initial write in my java program occurs? My config file is as follows:

input{
 file{
        path => "/root/test_last/complete/example.json"
        codec => "json"
        start_position => "beginning"
        ignore_older => 0
 }
}
filter     {
    mutate   {
            gsub => [ "message","\[",""]
            gsub => [ "message","\n",""]
            gsub => [ "event","\},\{",","]
        }
    json { source => message }
}

output{
    stdout { codec => rubydebug }
}
2

'ignore_older => 0' says to ignore any files more than zero seconds old, which is all files. You should remove this option.

When you say you are writing an array to the file does that mean you are appending to it?

It is unlikely you want both a json codec and a json filter. And if you have a json codec on a file input you do not have a field called message, so your filters will have no effect.

3

Just a singular array containing a bunch of json objects.

4

OK, so if you remove the ignore_older option does it work as expected?

5

I start getting this error in terms of parsing:

exception=>#<LogStash::Json::ParserError: Unexpected character (':' (code 58)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')

6

Your JSON is not valid JSON. For example, this will produce that error

[ { "foo" :: 1 } ]
7

Hm, thats interesting because I put my json through a jsonlint to validate it and it said it was a valid json...

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.