# Logstash parsing @timestamp even before the filter is triggered

**URL:** <https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [July 1, 2022, 11:50pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700 "2022-07-01T23:50:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rakesh\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rakesh_b/32/48128_2.png) [@Rakesh\_B](https://discuss.elastic.co/u/Rakesh_B)\
**Post date:** [July 1, 2022, 11:50pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700/1 "2022-07-01T23:50:04Z")

</div>

Hi,

- We are noticing that logstash is parsing the `@timestamp` field even before the `filter{}` is triggered and this is causing the warning `Error parsing @timestamp string value` and spamming our logs
- We push logs from fluent-bit --\> logstash --\> elasticsearch

How to reproduce:

- setup a docker-compose.yaml for elasticsearch and kibana
- now run a logstash server with the following pipeline

```auto
input {
  http {
    port => 8080
  }
}
filter {
  if [@timestamp] {
      mutate {
        remove_field => ["@timestamp"]
      }
    ruby {
    code => "event.set('logstash_processed_at', Time.now());"
    }

    mutate {
    convert => { "logstash_processed_at" => "string" }
    }

    date {
      match => ["logstash_processed_at", "ISO8601"]
      target => "@timestamp"
      add_tag => "timestamp_changed"
    }
  }
}
output {
  elasticsearch {
    ssl_certificate_verification => false
    hosts => ["elasticsearch:9200"]
    manage_template => true
    template_name => "k8s"
    index => "test"
    user => "elastic"
    password => "PASSWORD_HERE"
    codec => "json"
  }
}

```

- now setup a fluent-bit container which pushes logs to logstash server

```auto
[SERVICE]
    Log_Level info

[INPUT]
    Name dummy
    Dummy {"@message":"HTTP GET /prometheus","@timestamp":"2022-07-01 20:38:53"}
[OUTPUT]
    Name http
    Host logstash
    Port 8080
    Format json

```

that's when we notice that logstash server is throwing warning `[2022-07-01T23:44:36,496][WARN][org.logstash.Event][main][0f6b528c835b85f2aeef2fb1488d39cbb85d83d9b67ac664cabd98dae739f089] Error parsing @timestamp string value=2022-07-01 20:38:53` and as you can see from the pipeline config, we are removing the existing `@timestamp` field and updating with a new value.

You can also see in the below screenshot that the field added above `logstash_processed_at` exists from the logs

 ![Screen Shot 2022-07-01 at 4.48.51 PM](https://us1.discourse-cdn.com/elastic/original/3X/e/4/e4da6992fad84c15d538548c9ad7732d110984e0.jpeg)

Basically we want to ignore the `@timestamp` field IF it exists in the logs and use the current time and not see that `Error parsing @timestamp` warning logs.

- Thank you

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 2, 2022, 1:01am UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700/2 "2022-07-02T01:01:49Z")

</div>

> [@Rakesh\_B](#):
>
> ```auto
> [OUTPUT]
> Name http
> Host logstash
> Port 8080
> Format json
> 
> ```

If `Format json` cause the post into the http input to have Content-Type: application/json, then the input will run the POST body through a json codec. When the input tries to create the event from the parsed JSON it is getting an exception down inside [initTimestamp](https://github.com/elastic/logstash/blob/120648abe4ddbe824048204babeb9ebefd3492aa/logstash-core/src/main/java/org/logstash/Event.java#L111). Apparently "2022-07-01 20:38:53" is not a supported date format in that context.

You could set `additional_codes => {}` to override the default setting on the input, then use something like

```
mutate { gsub => ["message", '"@timestamp"\s*:\s*"[^"]*"', "" ] }

```

to strip the timestamp out, then use a json filter to parse the JSON.

Alternatively, try

```
Dummy {"@message":"HTTP GET /prometheus","@timestamp":"2022-07-01T20:38:53.000Z"}

```

---

<div class="post-metadata">

**Author:** ![Rakesh\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rakesh_b/32/48128_2.png) [@Rakesh\_B](https://discuss.elastic.co/u/Rakesh_B)\
**Post date:** [July 5, 2022, 5:09pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700/3 "2022-07-05T17:09:31Z")

</div>

Thank you, that makes sense.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2022, 5:10pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-even-before-the-filter-is-triggered/308700/4 "2022-08-02T17:10:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.