Filebeat system and iptables module timezone offset issue

[mback2k (Marc)]

Hello everyone,

I am using filebeat 6.7.1 with the system, iptables, traefik and apache2 modules. For some reason I am having issues with @timestamp being incorrect for system (from /var/log/syslog* and /var/log/auth.log*) and iptables (from /var/log/kern.log*) logs.

These logs are written with timestamps according to the timezone CET / CEST (Europe/Berlin), currently UTC+2. Filebeat is running in a container with the same timezone information. date gives the same date, time and timezone on the host and in the container.

For some reason the @timestamp is neither UTC nor UTC+2, but unfortunately rather UTC+4. This happens regardless of convert_timezone being true or false. This option only affects beat.timezone being 00:00 or 02:00.

For the other log files, like apache2 and traefik this issue does not happen, because the log lines already contain the timezone information in the timestamps.

Does anyone have any idea about this issue or experienced the same? What am I doing wrong, besides unfortunately not using UTC for all log files and systems.

Thanks and best regards,
Marc

[jsoriano (Jaime Soriano)]

Hi @mback2k,

After changing the value of var.convert_timezone you need to setup the pipelines again. You can do it by restarting filebeat with filebeat.overwrite_pipelines: true in the configuration. Take into account that this option can override other pipelines that you have manually setup. You may want to remove this option once the pipelines are setup as you expect.

You can check if the pipelines are being correctly configured by executing GET _ingest/pipeline/filebeat-6.7.1-system-* and checking that the date processors include the timezone option, like in:

        "date" : {
          "field" : "system.syslog.timestamp",
          "target_field" : "@timestamp",
          "formats" : [
            "MMM  d HH:mm:ss",
            "MMM dd HH:mm:ss"
          ],
          "timezone" : "{{ beat.timezone }}",
          "ignore_failure" : true
        }

Regards,
Jaime.

[jsoriano (Jaime Soriano)]

There is an open issue to allow to use filebeat setup --pipelines to reconfigure the pipelines if a setting is changed: https://github.com/elastic/beats/issues/9747

[mback2k (Marc)]

Hi @jsoriano,

thank you very much. That seems to have solved the problem for me even though

"timezone" : "{{ beat.timezone }}",

was already included inside the pipeline. I could not spot any difference after overwriting the pipelines, but it started working anyway.

Best regards,
Marc