Data Logstash filter not putting month into @timestamp

[adwearys]

Hi,

I'm trying to take the date and time from a syslog message and put it into the @timestamp field (so the @timestamp field has the date and time that the source system created the message, rather than when it arrived at the Logstash server). I've got this config:

filter {
  kv {
    id => "PCS-WELF"
  }
  date {
#   time="%date% %time%", e.g. time="2021-06-25 09:15:19"
    match => [ "time", "yyyy-MM-DD HH:mm:ss" ]
    timezone => "Europe/London"
  }
}

Most of the date and time make it into @timestamp, but not the month (!?) or timezone. For example, looking at the index in Kibana I see logs like this:

{
  "_index": "pcs-2021.01",
  "_type": "_doc",
  "_id": "tEuvQ3oBGJCdTLc2RUjz",
  "_version": 1,
  "_score": 0,
  "fields": {
[snip]  
    "@timestamp": [
      "2021-01-25T15:59:57.000Z"
    ],
    "time.keyword": [
      "2021-06-25 15:59:57"
    ],
[snip]
    "time": [
      "2021-06-25 15:59:57"
    ],
[snip]
  }
}

Any ideas why the @timestamp month is "01" (January) instead of "06" (June), and why the Europe/London timezone hasn't made it into @timestamp either? We're using Logstash version 7.13.2.

Cheers,
Alastair

[Badger]

@adwearys

Any ideas why the @timestamp month is "01" (January) instead of "06" (June), and why the Europe/London timezone hasn't made it into @timestamp either?

DD is day of the year, so 06 is Jan 6th and that overrides the month. Use dd instead.

Europe/London is not on BST in January so the offset from UTC is zero.

[adwearys]

Brilliant, thanks for spotting that. I'd changed "YYYY" to "yyyy" but was completely blind to the "d" and "dd" in Date filter plugin | Logstash Reference [7.13] | Elastic. Maybe it was from also looking at the Kibana date display format, which uses the Moment.js format pattern (Default: MMM D, YYYY @ HH:mm:ss.SSS).

All good now, e.g.:

@timestamp
	2021-06-25T16:40:20.000+01:00
time
	2021-06-25 16:40:20
 	Multi fields
 	time.keyword: 2021-06-25 16:40:20

Thanks again for your help!