# Timestamp field reordering events from patterns?

**URL:** <https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249>\
**Category:** Logstash\
**Created:** [October 7, 2020, 9:37am UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249 "2020-10-07T09:37:15Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 7, 2020, 9:37am UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/1 "2020-10-07T09:37:15Z")

</div>

Hi,

Yesterday i cam across something i can't understand.

Let's say i stopped my logstash at 10 AM and put it back on at 12, usually logs are coming and re-ordered as normal ( i can't figure how since @timestamp field should be ingest time ). As normal i mean it's near their creation time and everything is put back in the timeline.

Yesterday my pattern had a conflict or missing field and when i started my logstash every logs were spiking and @timestamp at the time i started my logstash thus creating a false timeline.

I tried to reproduce it but it looks like i am missing some pieces of informations.

My logs are shipped through filebeat to logstash then elastic.

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 7, 2020, 2:45pm UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/2 "2020-10-07T14:45:51Z")

</div>

I've tried to :

- Set mapping to type and format date to try to replace @timestamp in the pattern with event.created time or something more accurate to keep track of the event timeline.

```auto
PUT index
{
  "mappings": {
    "properties": {
      "datedate": {
        "type": "date" ,
        "format": "YYYY MM DD HH:mm:ss"
      }
    }
  }
}

```

Combining this to a logstash filter with the correct date format :

```auto
   mutate {
                                add_field => { "datedate" => "2020 10 07 16:01:01" }
                        }

                        date {
                                match => ["datedate", "YYYY MM dd HH:mm:ss"]
                        }

```

Seemed to work but my Kibana refused to understand the first part of the year depsite the configuration of the date format of kibana in the settings ( note that "dd" in logstash is not "dd" in Kibana )

And the final JSON doc :

```auto
"datedate": "2020 10 07 16:01:01",
"fields": {
    "datedate": [
      "2019-12-30T16:01:01.000Z"
    ],
    "@timestamp": [
      "2020-10-07T14:01:01.000Z"
    ]
  }, 

```

The original field in my doc is still shown as 2020 10 07 but Kibana created another from the pattern i believe with another value.

I dont know how all this works and i'd like to find answers on how the @timestamp field behave and how can we improve our configuration.

From what i've always understand @timestamp is from the ingestion time but i've seen it re-ordering logs in the correct time after a production crash or something.

I'd like to understand how all this is working.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 7, 2020, 3:18pm UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/3 "2020-10-07T15:18:23Z")

</div>

> [@grumo35](#):
>
> From what i've always understand @timestamp is from the ingestion time

By default, logstash adds the @timestamp field with the current time (i.e. when it is ingested). The date filter will overwrite this with whatever it parses. @timestamp is always in UTC, so if you are two hours ahead of UTC a date filter will convert "2020 10 07 16:01:01" to 2020-10-07T14:01:01.000Z

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 7, 2020, 3:22pm UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/4 "2020-10-07T15:22:24Z")

</div>

> By default, logstash adds the @timestamp field with the current time

What i can't understand is how this value is changed to a time were logstash was down ?

And why the date filter is changing the Year of the field in Kibana from 2020 10 07 to 2019 12 30 ?

---

<div class="post-metadata">

**Author:** ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)\
**Post date:** [October 7, 2020, 3:33pm UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/5 "2020-10-07T15:33:45Z")

</div>

Sorry if i'm not very clear on the issue,

First on pattern X :

 ![logstash](https://us1.discourse-cdn.com/elastic/original/3X/5/1/51b80d510b97d2e7edb366628a3a218783b7ccf3.png)  
Then on pattern Y:  
 ![Sélection_004](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb58f740ae90261fa7ecbde40ec1d05135c32549.png)

based on @timestamp definition this happening is kinda weird ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2020, 3:33pm UTC](https://discuss.elastic.co/t/timestamp-field-reordering-events-from-patterns/251249/6 "2020-11-04T15:33:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.