# Logstash parsing timestamp incorrectly

**URL:** <https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773>\
**Category:** Logstash\
**Created:** [August 12, 2020, 6:24pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773 "2020-08-12T18:24:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![anandd4](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anandd4/32/50610_2.png) [@anandd4](https://discuss.elastic.co/u/anandd4)\
**Post date:** [August 12, 2020, 6:24pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773/1 "2020-08-12T18:24:27Z")

</div>

**Problem Statement -**  
Filebeat streaming logs via logstash into elasticsearch (7.5)  
Logstash applies the filter on epoch time & update @timestamp field during data ingestion  
Logstash is incorrectly ingesting the timestamp field for some records during ingestion. In short its trimming the epoch time field which creates inconsistency.

**logs snippet -**

```
1597161900000,host1,2,Request,proc,Process001,0,0,0.000000,0.000000,0.000000,0.000000,0
1597161900000,host1,1,Request,proc,Process001,0,0,0.000000,0.000000,0.000000,0.000000,0
1597161900000,host1,1,Request,proc,Process001,0,0,0.000000,0.000000,0.000000,0.000000,0
1597161900000,host1,1,Request,proc,Process001,0,0,0.000000,0.000000,0.000000,0.000000,0
1597161900000,host1,1,Request,proc,Process001,0,0,0.000000,0.000000,0.000000,0.000000,0

```

**logstash conf -**

```
  dissect {
    mapping => {
    "message" => "%{timestamp},%{keyvalue00},%{keyvalue01},%{keyvalue02},%{keyvalue03},%{keyvalue04},%{keyvalue05},%{keyvalue06},%{keyvalue07},%{keyvalue08},%{keyvalue09},%{keyvalue10},%{keyvalue11}"
    }
  remove_field => "message"
  }
	  date {
    match => ["timestamp", "UNIX_MS"]
    target => "@timestamp"
  }

```

**Expected Behaviour** -  
All timestamps should be correctly parsed during data ingestion to elasticsearch

**Actual Behaviour** -  
Logstash is trimming the timestamp field during ingestion. e.g. 1597161900000 is ingested as 61900000 or 597161900000. This creates inconsistency & a no. of records are not ingested to the targeted index as intended.  
PS - This behaviour is rare but does happen once in couple of days & there is no change at the source.

Can someone please explain what could be causing this unexpected behaviour & how to mitigate it?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2020, 7:00pm UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773/2 "2020-08-12T19:00:32Z")

</div>

I very much doubt logstash is doing that. I think it more likely that you are seeing [inode re-use](https://www.elastic.co/guide/en/beats/filebeat/current/inode-reuse-issue.html) causing filebeat to start reading a file part way through.

---

<div class="post-metadata">

**Author:** ![anandd4](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anandd4/32/50610_2.png) [@anandd4](https://discuss.elastic.co/u/anandd4)\
**Post date:** [August 27, 2020, 8:47am UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773/3 "2020-08-27T08:47:05Z")

</div>

Thanks for pointing in the right direction. Its sorted

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 24, 2020, 8:47am UTC](https://discuss.elastic.co/t/logstash-parsing-timestamp-incorrectly/244773/4 "2020-09-24T08:47:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.