Filebeats input json_lines only processing single entry


(Werring) #1

I am just starting with logstash (and the other products of the elastic stack) but im having some issues with trying to set up my sample parser.
Only the first line of my log is sent to the stdout when using the json_lines codec.
When i delete the codec line, all log entries are sent to my stdout (but not parsed as json).

Any suggestion in the right direction would be welcome!

My logstash config:

input {
    beats { 
        port => 5044
        codec => json_lines {}
    }
}

output {
    stdout { codec => rubydebug }
}

My sample log:

{"cocoon":{"name":"Blub","url":"https://blub.example.org/","version":"2.12.1 beta","type":"demo"},"file":{"type":"image/jpeg","extension":"jpg","name":"20180917_201321.jpg","variant":"2000px","filesize":1013002},"download":261320,"date":"2018-08-02T07:22:07.000"}
{"cocoon":{"name":"Team","url":"https://team.example.org/","version":"2.12.1 beta","type":"demo"},"file":{"type":"image/gif","extension":"gif","name":"styleguide.gif","variant":"2000px","filesize":717281},"download":547680,"date":"2018-08-27T09:51:01.000"}
{"cocoon":{"name":"Blub","url":"https://blub.example.org/","version":"2.12.1 beta","type":"demo"},"file":{"type":"application/zip","extension":"zip","name":"roos.zip","variant":"original","filesize":1862406},"download":815697,"date":"2018-03-17T23:48:02.000"}
{"cocoon":{"name":"Blub","url":"https://blub.example.org/","version":"2.12.1 beta","type":"demo"},"file":{"type":"image/jpeg","extension":"jpg","name":"logo.jpg","variant":"web","filesize":821324},"download":616235,"date":"2018-06-09T04:46:31.000"}
{"cocoon":{"name":"Doh","url":"https://doh.example.org/","version":"2.12.2 alpha","type":"dev"},"file":{"type":"image/png","extension":"png","name":"presskit-2007.png","variant":"web","filesize":869585},"download":429289,"date":"2018-05-29T02:20:40.000"}

Logstash output:

{
      "@version" => "1",
          "date" => "2018-08-02T07:22:07.000",
      "download" => 261320,
          "file" => {
          "variant" => "2000px",
             "type" => "image/jpeg",
         "filesize" => 1013002,
        "extension" => "jpg",
             "name" => "20180917_201321.jpg"
    },
        "cocoon" => {
           "type" => "demo",
            "url" => "https://blub.example.org/",
           "name" => "Blub",
        "version" => "2.12.1 beta"
    },
    "@timestamp" => 2018-10-22T07:04:29.705Z
}

(Christian Dahlqvist) #2

What if you use a json codec instead of json_lines?


(Werring) #3

Oh wow, that worked!
Thanks!


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.