Continuing the discussion from @timestamp field not matching with the Actual log field:
Hi, I have been using ELK for quite a sometime now. My logs are in following pattern
May 9 03:45:25 rjedi-04 docker/3fdf29fdc1eb[8054]: 2016-05-09 03:45:25,940 [main] INFO ly.stealth.mesos.kafka.HttpServer$ - started on port 4239
my logstash conf is
grok {
match => { "message" => "%{CISCOTIMESTAMP:syslog_timestamp} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{URIHOST:date_value} %{TIME:time_value}%{GREEDYDATA:syslog_message}" }
add_field => { "primary_timestamp" => "%{date_value} %{time_value}" }
add_field => { "is_matched" => "true" }
add_field => { "received_from" => "%{host}" }
add_field => { "is_primary_tstmp" => "true" }
}
date {
match => [ "primary_timestamp", "YYYY-MM-dd HH:mm:ss','SSS", "YYYY-MM-dd HH:mm:ss'.'SSS", "YYYY-MM-dd HH:mm:ss':'SSS" ]
target => "@timestamp"
remove_field => [ "time_value", "date_value", "primary_timestamp" ]
}
if "_dateparsefailure" in [tags] {
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
target => "@timestamp"
}
}
When ever Logstash parses the logs @timestamp value is 2016-05-09T03:45:25.940Z
{
"message" => "May 9 03:45:25 rjedi-04 docker/3fdf29fdc1eb[8054]: 2016-05-09 03:45:25,940 [main] INFO ly.stealth.mesos.kafka.HttpServer$ - started on port 4239",
"@version" => "1",
"@timestamp" => "2016-05-09T03:45:25.940Z",
"host" => "test.hello.com",
"syslog_timestamp" => "May 9 03:45:25",
"syslog_hostname" => "rjedi-04",
"syslog_program" => "docker/3fdf29fdc1eb",
"syslog_pid" => "8054",
"syslog_message" => " [main] INFO ly.stealth.mesos.kafka.HttpServer$ - started on port 4239",
"is_matched" => "true",
"received_from" => "test.hello.com",
"is_primary_tstmp" => "true"
}
However same is not reflected on Kibana , in kibana it is showing as May 9th 2016, 09:15:25.940
All of my ELK stack is in UTC timezone. I am unable to find where is it being changed, as Kibana accepts UTC timestamp, and ideally it shouldn't modify, correct me if i am wrong.