I'm stuck trying to get my Ingres log into ElasticSearch using logstash. Specifically I want to use the date in my logs to be the @timestamp.
Here is sample log data I'm trying to process:
::[ingres , 000020ed]: Thu Dec 29 14:20:41 2005 E_CL2530_CS_PARAM session_accounting = OFF
::[ingres , 000020ed]: Thu Dec 29 14:20:41 2005 E_CL2530_CS_PARAM stack_size = 131072
Here is my logstash conf file I'm using, I've tried piles of combinations but can't quite get there. My grok match line seems to work to parse it into the individual fields, but then I can't get the date filter to work. As you can see I've tried using add_field but the %{logyear} doesn't seem to work, which was a test to see if I could piece together a new field from all the individual ones, and then use this new field in the date filter.
input {
file {
path => "C:\Logs\Ingres\Dev\ingreserrlog.log"
type => "ingres"
start_position => "beginning"
}
}
filter {
if [type] == "ingres" {
grok {
match => [ "message", "(?.)::(?[a-zA-Z0-9, ._[]]+): +%{WORD:dayofweek} +(?%{WORD:logmonth} +%{WORD:logdayofmonth} +%{WORD:loghour}:%{WORD:logminute}:%{WORD:logsecond} +%{WORD:logyear}) +%{WORD:ingreserrorcode}\s%{GREEDYDATA:ingreserrormessage}" ]
}
mutate {
add_field => { "logdate" => "%{@logyear}-Jan-23 11:42:09.123" }
}
date {
match => [ "logdate", "yyyy-MMM-dd HH:mm:ss.SSS" ]
locale => "en"
timezone => "UTC"
}
}
}
output {
elasticsearch {
hosts => ["192.168.7.103:9200"]
index => "ingresdevelopment"
}
stdout {}
}