Logstash wont add date to logs


(Michael) #1

How can I get logstah add the date field that I have added to my jboss logs?
The log looks like:
August 10 2015 10:07:01,048 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 38) JBAS011800: Activating Naming Subsystem

My indexer is:
input {
redis {
host => "XXX.XXX.XXX.XXX"
port => 6379
data_type => "list"
key => "logstash"
codec => "json"
}
}
filter {
if [type] == "jboss" {
grok {
match => [
"message",
"%{TIME:time} %{LOGLEVEL:level}.*[(?[^]]+)] ((?[^)]+)) %{GREEDYDATA:message}"
]
overwrite => ["message"]
}
}
multiline {
type => "jboss"
pattern => "^\s"
what => "previous"
}
}
output {
elasticsearch {
host => "XXX.XXX.XXX.XXX"
port => 9300
}
}

I need the time field to show both the log date and the time.


(Magnus Bäck) #2

Use a date filter to parse the time field and make sure it includes the date too when you create the field with the grok filter.


(Michael) #3

I'm not sure how to write it or where in the grok filter to put it.


(Magnus Bäck) #4

Well, change the beginning of your grok expression like this:

grok {
  match => [
    "message",
    "(?<time>%{MONTH} %{MONTHDAY} %{YEAR} %{TIME}) %{LOGLEVEL:level} ..."
  ]
}

This should leave you with a time field containing e.g. "August 10 2015 10:07:01,048". A date filter to parse this probably looks like this:

date {
  match => ["time", "MMM dd YYYY HH:mm:ss,SSS"]
  remove_field => ["time"]
}

I also suggest that you place the multiline filter first (followed by the grok and date filters).


(system) #5