# Filebeat 7.3.2 logstash module timezone bug - Time in the future

**URL:** <https://discuss.elastic.co/t/filebeat-7-3-2-logstash-module-timezone-bug-time-in-the-future/203900>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 16, 2019, 5:34pm UTC](https://discuss.elastic.co/t/filebeat-7-3-2-logstash-module-timezone-bug-time-in-the-future/203900 "2019-10-16T17:34:41Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Khatarian](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khatarian/32/56041_2.png) [@Khatarian](https://discuss.elastic.co/u/Khatarian)\
**Post date:** [October 16, 2019, 5:34pm UTC](https://discuss.elastic.co/t/filebeat-7-3-2-logstash-module-timezone-bug-time-in-the-future/203900/1 "2019-10-16T17:34:41Z")

</div>

Hey there!

For awhile I noticed that logs from the Logstash module are in the future so I went digging in this bug.

I'm using logstash to receive the logs and forward them to the ingress pipeline of elasticsearch. To fix this I changed the ingest pipeline by doing the following:

```
DELETE /_ingest/pipeline/filebeat-7.3.2-logstash-log-pipeline-plain

PUT /_ingest/pipeline/filebeat-7.3.2-logstash-log-pipeline-plain
{
    "description": "Pipeline for parsing logstash logs in the plain format",
    "on_failure": [
        {
            "set": {
                "field": "error.message",
                "value": "{{ _ingest.on_failure_message }}"
            }
        }
    ],
    "processors": [
        {
            "grok": {
                "field": "message",
                "pattern_definitions": {
                    "LOGSTASH_CLASS_MODULE": "[\\w\\.]+",
                    "LOGSTASH_LOGLEVEL": "INFO|ERROR|DEBUG|FATAL|WARN|TRACE",
                    "GREEDYMULTILINE" : "(.|\n)*"
                },
                "patterns": [
                    "\\[%{TIMESTAMP_ISO8601:logstash.log.timestamp}\\]\\[%{LOGSTASH_LOGLEVEL:log.level}\\s?\\]\\[%{LOGSTASH_CLASS_MODULE:logstash.log.module}\\s*\\] %{GREEDYMULTILINE:message}"
                ]
            }
        },
        {
            "rename": {
                "field": "@timestamp",
                "target_field": "event.created"
            }
        },
        {
            "date": {
                "field": "logstash.log.timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "yyyy-MM-dd'T'HH:mm:ss,SSS"
                ],
                "ignore_failure": true
            }
        },
        {
            "date": {
                "if": "ctx.event.timezone != null",
                "field": "logstash.log.timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "yyyy-MM-dd'T'HH:mm:ss,SSS"
                ],
                "timezone": "{{ event.timezone }}",
                "on_failure": [{"append": {"field": "error.message", "value": "{{ _ingest.on_failure_message }}"}}]
            }
		},	
        {
            "remove": {
                "field": "logstash.log.timestamp"
            }
        }
    ]
}

```

And for the slowlogs:

```
DELETE /_ingest/pipeline/filebeat-7.3.2-logstash-slowlog-pipeline-plain

PUT /_ingest/pipeline/filebeat-7.3.2-logstash-slowlog-pipeline-plain
{
    "description": "Pipeline for parsing logstash slowlogs in the plain format",
    "on_failure": [
        {
            "set": {
                "field": "error.message",
                "value": "{{ _ingest.on_failure_message }}"
            }
        }
    ],
    "processors": [
        {
            "grok": {
                "field": "message",
                "pattern_definitions": {
                    "LOGSTASH_CLASS_MODULE": "[\\w\\.]+\\s*",
                    "LOGSTASH_LOGLEVEL": "INFO|ERROR|DEBUG|FATAL|WARN|TRACE"
                },
                "patterns": [
                    "\\[%{TIMESTAMP_ISO8601:logstash.slowlog.timestamp}\\]\\[%{LOGSTASH_LOGLEVEL:log.level}\\s?\\]\\[%{LOGSTASH_CLASS_MODULE:logstash.slowlog.module}\\] %{GREEDYDATA:message}"
                ]
            }
        },
        {
            "grok": {
                "field": "logstash.slowlog.module",
                "patterns": [
                    "slowlog.logstash.%{WORD:logstash.slowlog.plugin_type}.%{WORD:logstash.slowlog.plugin_name}"
                ]
            }
        },
        {
            "grok": {
                "field": "message",
                "patterns": [
                    "{:plugin_params=>%{GREEDYDATA:logstash.slowlog.plugin_params}, :took_in_nanos=>%{NUMBER:event.duration}, :took_in_millis=>%{NUMBER:logstash.slowlog.took_in_millis}, :event=>%{GREEDYDATA:logstash.slowlog.event}}"
                ]
            }
        },
        {
            "rename": {
                "field": "@timestamp",
                "target_field": "event.created"
            }
        },
                {
            "date": {
                "field": "logstash.slowlog.timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "yyyy-MM-dd'T'HH:mm:ss,SSS"
                ],
                "ignore_failure": true
            }
        },
        {
            "date": {
                "if": "ctx.event.timezone != null",
                "field": "logstash.slowlog.timestamp",
                "target_field": "@timestamp",
                "formats": [
                    "yyyy-MM-dd'T'HH:mm:ss,SSS"
                ],
                "timezone": "{{ event.timezone }}",
                "on_failure": [{"append": {"field": "error.message", "value": "{{ _ingest.on_failure_message }}"}}]
            }
		},
        {
            "remove": {
                "field": [
                    "message",
                    "logstash.slowlog.timestamp"
                ]
            }
        },
        {
            "convert": {
                "field": "event.duration",
                "type": "long"
            }
        },
        {
            "convert": {
                "field": "logstash.slowlog.took_in_millis",
                "type": "long"
            }
        }
    ]
}

```

This fixes it, Not sure what the ingest-json.json file does, but I didn't edit those ones to make it work. All I changed was how the date is handled in the ingest. I hope it's useful to someone and maybe someday it could make it in a new release!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 13, 2019, 5:34pm UTC](https://discuss.elastic.co/t/filebeat-7-3-2-logstash-module-timezone-bug-time-in-the-future/203900/2 "2019-11-13T17:34:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.