Filebeat 7.3 syslog / auth log timezone parsing error

[Kent_Wang]

Hi Guys,

I think there is something wrong in the system module ingest pipeline related to timezone processing.

Here is an excerpt of date processors from pipipeline.json:

{
    "date": {
        "field": "system.auth.timestamp",
        "target_field": "@timestamp",
        "formats": [
            "MMM  d HH:mm:ss",
            "MMM dd HH:mm:ss",
            "ISO8601"
        ],
        "ignore_failure": true
    }
},
{
    "date": {
        "if": "ctx.event.timezone != null",
        "field": "@timestamp",
        "formats": ["ISO8601"],
        "timezone": "{{ event.timezone }}",
        "on_failure": [{"append": {"field": "error.message", "value": "{{ _ingest.on_failure_message }}"}}]
    }
},

When the first date processor is executed, the time field is parsed in UTC timezone, then
the second date processor just adjust @timestamp to event.timezone, which is still wrong.

For example, assume that we have a server in timezone +08:00, and system.auth.timestamp is "Aug 21 11:15:01":

1. the first date processor parses system.auth.timestamp in UTC timezone, which got:
   {
   "@timestamp": "2019-08-21T11:15:01Z"
   }

   As you can see, the timestamp is not right, it's 8 hours ahead.

2. the second date processor parses @timestamp field in +08:00 timezone, which got:
   {
   "@timestamp": "2019-08-21T19:15:01+08:00"
   }

   The time point is not changed, still 8 hours ahead, just represented in another timezone.

I have a fix here. If it's confirmed as a bug, I'll submit a github issue and pull request.

Fix filebeat system module timezone parsing

[faec (Fae Charlton)]

Welcome! The syslog timestamp handling is rather confusing, as it can be affected by several settings. Can you share an accompanying filebeat.yml that exhibits the problem?

[Kent_Wang]

Hi, it's been confirmed as a bug.

https://github.com/elastic/beats/pull/13308