Filebeat 7.3 syslog / auth log timezone parsing error

Hi Guys,

I think there is something wrong in the system module ingest pipeline related to timezone processing.

Here is an excerpt of date processors from pipipeline.json:

{
    "date": {
        "field": "system.auth.timestamp",
        "target_field": "@timestamp",
        "formats": [
            "MMM  d HH:mm:ss",
            "MMM dd HH:mm:ss",
            "ISO8601"
        ],
        "ignore_failure": true
    }
},
{
    "date": {
        "if": "ctx.event.timezone != null",
        "field": "@timestamp",
        "formats": ["ISO8601"],
        "timezone": "{{ event.timezone }}",
        "on_failure": [{"append": {"field": "error.message", "value": "{{ _ingest.on_failure_message }}"}}]
    }
},

When the first date processor is executed, the time field is parsed in UTC timezone, then
the second date processor just adjust @timestamp to event.timezone, which is still wrong.

For example, assume that we have a server in timezone +08:00, and system.auth.timestamp is "Aug 21 11:15:01":

  1. the first date processor parses system.auth.timestamp in UTC timezone, which got:
    {
    "@timestamp": "2019-08-21T11:15:01Z"
    }

    As you can see, the timestamp is not right, it's 8 hours ahead.

  2. the second date processor parses @timestamp field in +08:00 timezone, which got:
    {
    "@timestamp": "2019-08-21T19:15:01+08:00"
    }

    The time point is not changed, still 8 hours ahead, just represented in another timezone.

I have a fix here. If it's confirmed as a bug, I'll submit a github issue and pull request.

Welcome! The syslog timestamp handling is rather confusing, as it can be affected by several settings. Can you share an accompanying filebeat.yml that exhibits the problem?

Hi, it's been confirmed as a bug.

https://github.com/elastic/beats/pull/13308

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.