Logstash json parsing error to read logback json file

[Juneyoung_Oh]

Hi. I am trying to read and store tomcat logs into elasticsearch via logstash.
The json file which made by logback looks like below.

{
  "@timestamp" : "2017-06-08T09:33:48.579+09:00",
  "@version" : 1,
  "message" : "FrameworkServlet 'dispatcherServlet': initialization completed in 1851 ms",
  "logger_name" : "org.springframework.web.servlet.DispatcherServlet",
  "thread_name" : "http-nio-8090-exec-1",
  "level" : "INFO",
  "level_value" : 20000,
  "HOSTNAME" : "Juneui-MacBook-Pro.local"
}
{
  "@timestamp" : "2017-06-08T09:33:48.631+09:00",
  "@version" : 1,
  "message" : "index() called, hits was 'null', session id '94DCBEF7DF26440F321F083D574EC747'",
  "logger_name" : "org.owls.session.SessionTestController",
  "thread_name" : "http-nio-8090-exec-1",
  "level" : "INFO",
  "level_value" : 20000,
  "HOSTNAME" : "Juneui-MacBook-Pro.local"
}

and my logstash config like

input {
  file {
    codec => json
    type => json
    path => '/etc/logstash/logs/log*.json'
  }
}

filter {
  json {
    source => 'message'
  }
}

output {
  elasticsearch {
    hosts => ['localhost:9200']
    index => 'elk-%{+YYYY.MM.dd}'
    document_type => 'log'
  }
  stdout { codec => rubydebug }
}

I put type and codec, when I watch the logstash log, it prints jsonparse failure. I also tried input codec => json_lines, but did not work well.

Here is logstash error log for your information,

{
          "path" => "/etc/logstash/logs/log.json",
    "@timestamp" => 2017-06-08T01:36:41.467Z,
      "@version" => "1",
          "host" => "f49930138437",
       "message" => "}",
          "type" => "json",
          "tags" => [
        [0] "_jsonparsefailure"
    ]
}

How can I fix this?

[magnusbaeck (Magnus Bäck)]

You'll have to use a multiline codec to join the lines of a JSON message into a single event. The multiline logic could be "unless the line is a lone opening brace, join the current line with the previous line".