Hi Guys,
I think there is something wrong in the system module ingest pipeline related to timezone processing.
Here is an excerpt of date processors from pipipeline.json:
{
"date": {
"field": "system.auth.timestamp",
"target_field": "@timestamp",
"formats": [
"MMM d HH:mm:ss",
"MMM dd HH:mm:ss",
"ISO8601"
],
"ignore_failure": true
}
},
{
"date": {
"if": "ctx.event.timezone != null",
"field": "@timestamp",
"formats": ["ISO8601"],
"timezone": "{{ event.timezone }}",
"on_failure": [{"append": {"field": "error.message", "value": "{{ _ingest.on_failure_message }}"}}]
}
},
When the first date processor is executed, the time field is parsed in UTC timezone, then
the second date processor just adjust @timestamp to event.timezone, which is still wrong.
For example, assume that we have a server in timezone +08:00, and system.auth.timestamp is "Aug 21 11:15:01":
-
the first date processor parses system.auth.timestamp in UTC timezone, which got:
{
"@timestamp": "2019-08-21T11:15:01Z"
}As you can see, the timestamp is not right, it's 8 hours ahead.
-
the second date processor parses @timestamp field in +08:00 timezone, which got:
{
"@timestamp": "2019-08-21T19:15:01+08:00"
}The time point is not changed, still 8 hours ahead, just represented in another timezone.
I have a fix here. If it's confirmed as a bug, I'll submit a github issue and pull request.