Date difference in events received from filebeat


I am using filebeat on a remote linux machine to send events to logstash.

I am using grok and the date module to get events' timestamps.

Here is my complete configuration:

input {
  beats {
    type => "gemma_logs"
    port => 5045

  if [type] == "gemma_logs" {
    grok {
      match => ["message" , "%{TIMESTAMP_ISO8601:logdate} %{DATA:level} \[%{DATA:class}\] \(%{DATA:file}:%{NUMBER:line}\) - %{GREEDYDATA:log_message}"]
    date {
      match => [ "logdate", "YYYY-mm-dd HH:mm:ss,SSS" ]

output {
  if [type] == "gemma_logs" {
    elasticsearch {
      hosts => "localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"

This worked fine until January 31st 23:59:59 (?!). However, no index has been created for February until now (the 3rd). In the logstash debug logs I see that there is a strange time difference now between events and their matched timestamp; it is exactly 2 hours to a month:

Here is an example:

"message"=>"2019-02-03 14:24:59,147 DEBUG [cti.cdr.CdrLogger] 
"logdate"=>"2019-02-03 14:24:59,147",

Here is the complete event from filebeat:

[2019-02-03T14:25:04,547][DEBUG][logstash.pipeline        ] output received 
	{"event"=> {"source"=>"/var/log/gemma/telephonyserver.log",
								"version"=>"9 (stretch)",
				"log_message"=>"deliveredEvent(device: 302106960115)",
				"logdate"=>"2019-02-03 14:24:59,147",
				"message"=>"2019-02-03 14:24:59,147 DEBUG [cti.cdr.CdrLogger] ( - deliveredEvent(device: 302106960115)",

I'll appreciate any help or ideas.

Change YYYY-mm-dd to YYYY-MM-dd

That worked. Thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.