Trouble with json codec and/or date filter

Elastic Stack Logstash
1

Hi, I'm using ELK for centralized logging and I'm having trouble with a log. I just changed this log from plain text to json format. After I added the JSON codec, I can no longer see logs in kibana.

In the logstash debug logs I can see the logs reaching the input, filters, and output. I'm reformatting for clarity. Other logs from this box are making it to elasticsearch. I can't check the elasticsearch logs because I using the elastic cloud. I'm not sure where else to look.

input {
  file {
    type => "TomcatAccess"
    path => "/data02/logs/localhost_access_log.*"
    codec => json
  }
}
filter {
  if [type] == "TomcatAccess" {
    date {
      match => ["timestamp", "[dd/MMM/yyyy:HH:mm:ss ZZ]"]
      target => "@timestamp"
    }
  }
}
output {
    elasticsearch {
      hosts => "https://example.found.io:9243"
      user => "user"
      password => "password"
      index => "logs-%{+YYYY.MM.dd}"
    }
  }
}

{
  :timestamp=>"2017-11-07T15:21:49.152000-0800", 
  :message=>"Received line", 
  :path=>"/data02/logs/localhost_access_log.2017-11-07.txt", 
  :text=>"{\"host\": \"10.50.12.150\",  \"x-forwarded-for\": \"-\",  \"timestamp\": \"[07/Nov/2017:15:21:39 -0800]\",  \"method\": \"GET\",  \"path\": \"/App/testInitialized.cfm\",  \"process_time_ms\": \"49\",  \"status\": \"200\",  \"bytes_sent\": \"1218\",  \"referer\": \"-\"}", 
  :level=>:debug, 
  :file=>"logstash/inputs/file.rb", 
  :line=>"306", 
  :method=>"log_line_received"
}

{
  :timestamp=>"2017-11-07T15:21:49.159000-0800", 
  :message=>"output received", 
  :event=>{
    "host"=>"10.50.19.163", 
    "x-forwarded-for"=>"-", 
    "timestamp"=>"[07/Nov/2017:15:21:39 -0800]", 
    "method"=>"GET", 
    "path"=>"/App/testInitialized.cfm", 
    "process_time_ms"=>"52", 
    "status"=>"200", 
    "bytes_sent"=>"1218", 
    "referer"=>"-", 
    "@version"=>"1", 
    "@timestamp"=>"2017-11-07T23:21:39.000Z", 
    "type"=>"TomcatAccess", 
    "role"=>"app", 
    "pod"=>"pod2", 
    "environment"=>"dev", 
    "location"=>"oh"}, 
  :level=>:debug, 
  :file=>"(eval)", 
  :line=>"469", 
  :method=>"output_func"
}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.