Date match pattern with Z versus ZZ: any difference?


(Graham Hannington) #1

In brief: According to the docs, only ZZ matches +08:00. In practice, Z also matches. Why?

In detail...

From the Logstash 5.0 documentation for the match option of the date filter plugin:

The date formats allowed are anything allowed by Joda-Time (java time library). You can see the docs for this format here:
joda.time.format.DateTimeFormat

From that Joda-Time documentation:

Symbol  Meaning                      Presentation  Examples
------  -------                      ------------  -------
Z       time zone offset/id          zone          -0800; -08:00; America/Los_Angeles

The count of pattern letters determine the format.
Zone: 'Z' outputs offset without a colon, 'ZZ' outputs the offset with a colon, 'ZZZ' or more outputs the zone id.

I note that this Joda-Time documentation uses the verb "outputs" rather than "matches". I understand why, but I found it jarring to be pointed from Logstash documentation for match to Joda-Time documentation that refers to "outputting". Patterns for matching input and patterns for formatting output are not necessarily the same thing.

Here's an example of a time stamp from my data:

 2016-11-07 13:01:22.902+08:00

Note the format of the zone designator:

+08:00

That is, with a colon.

According to the Joda-Time documentation - if I interpret "outputs" in that documentation as "matches" - I need to specify ZZ at the end of a pattern to match the zone designator format I'm using.

However, in practice - specifically, with Logstash 1.5.5 (but the corresponding docs are the same) - the following date pattern, ending with a single Z, which I'd expect not to work:

date {
  match => ["time", "yyyy-MM-dd HH:mm:ss.SSSZ"]
}

appears to work just as well as a date pattern ending with ZZ.

I would have expected that the combination of offset-with-colon and Z would cause Logstash to report an error such as:

Failed parsing date from field {:field=>"time", :value=>"2016-11-07 13:01:22.902+08:00", :exception=>"Invalid format: "2016-11-07 13:01:22.902+08:00" is malformed at "+08:00"", :config_parsers=>"yyyy-MM-dd HH:mm:ss.SSSZ", :config_locale=>"default=en_US", :level=>:warn}

What's going on here?

What should I specify in my date pattern? Z or ZZ? It doesn't appear to make any difference. Am I missing something?


(Graham Hannington) #2

Logstash developers? Anyone?

I believe the current behavior contradicts the docs, but I'd be happy to be proven wrong.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.