Ingest Node Timezone Offset

Elastic Stack Elasticsearch
1

I've found an inconsistency between Logstash date matching syntax and Ingest node date matching syntax. I have a time field that looks like this:

2020-06-03 15:37:14.465 -04:00

While most of the Logstash Date Plugin syntax applies to the Ingest Node Date Processor, it doesn't look like the time zone syntax is the same. Based on the time zone match conditions, this match expression should work for the value above: yyyy-MM-dd HH:mm:ss.SSS ZZ. However, a processor with this match condition will fail with an illegal_argument_exception. The Logstash documentation clearly defines the time zone field:

ZZ
Timezone offset structured as HH:mm (colon in between hour and minute offsets). Example: -07:00.

z
time zone names. Time zone names ( z ) cannot be parsed.

But the only match condition that works is yyyy-MM-dd HH:mm:ss.SSS z. z should match time zone names, not offsets structured the way it's documented. Here's a simulation of it successfully executing:

Request

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "date": {
          "field": "timestamp",
          "formats": ["yyyy-MM-dd HH:mm:ss.SSS z"]
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "timestamp": "2020-06-03 15:37:14.465 -04:00"
      }
    }
  ]
}

Response

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "@timestamp" : "2020-06-03T19:37:14.465Z",
          "timestamp" : "2020-06-03 15:37:14.465 -04:00"
        },
        "_ingest" : {
          "timestamp" : "2020-06-05T20:11:29.730182Z"
        }
      }
    }
  ]
}

Is this a bug or is this expected behavior that didn't get documented?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.