# Low @timestamp precision cause wrong order of events with docker input

**URL:** <https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [June 22, 2018, 3:21pm UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051 "2018-06-22T15:21:04Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![smamontov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smamontov/32/32588_2.png) [@smamontov](https://discuss.elastic.co/u/smamontov)\
**Post date:** [June 22, 2018, 3:21pm UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/1 "2018-06-22T15:21:04Z")

</div>

Hi! I'm using Filebeat 6.3.0 and Docker 18.0.05-ce. Docker stores timestamps with nanoseconds, but elasticsearch date format precision is a millisecond.

> <https://github.com/elastic/elasticsearch/issues/10005>

Looks like this issue never be fixed.

Is there any elegant solutions nowadays? Maybe it will be OK to use string type in index mappings and just sort lexicographically? In this case, how to retrieve this field from docker log-json files?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [June 22, 2018, 3:29pm UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/2 "2018-06-22T15:29:42Z")

</div>

Have you considered to sort by offset as well. According to the [docs](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-log.html#exported-fields-log), it is the file offset. Problem is with file toration. Using ingest node or Logstash, one could try combine timestamp and offset into a 'sortable' number of type 'long'.

---

<div class="post-metadata">

**Author:** ![smamontov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smamontov/32/32588_2.png) [@smamontov](https://discuss.elastic.co/u/smamontov)\
**Post date:** [June 28, 2018, 10:16am UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/3 "2018-06-28T10:16:29Z")

</div>

Thanks for response! I know I can use offset for this, but in my setup I want log files to rotate often (there's some space limitations), so this doesn't look perfect. Is there a way to configure docker prospector to pass that precise time string (provided by json-log) in a separate field?

---

<div class="post-metadata">

**Author:** ![markdaku](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@markdaku](https://discuss.elastic.co/u/markdaku)\
**Post date:** [July 2, 2018, 8:57am UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/4 "2018-07-02T08:57:39Z")

</div>

When high precision is required the offset trick can work. But not always.

In a multi threaded application the log buffering may result in log events entering the log slightly un-ordered. But with a high precision timestamp this is not an issue.

I second that this be brought up as an enhancement moving forward.

---

<div class="post-metadata">

**Author:** ![pytimer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pytimer/32/29331_2.png) [@pytimer](https://discuss.elastic.co/u/pytimer)\
**Post date:** [July 10, 2018, 3:00am UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/5 "2018-07-10T03:00:25Z")

</div>

Hi, i also have this problem when i use Filebeat collect docker logs.

Because my project urgent previously and i unfamiliar `beats` code, so i fork [https://github.com/elastic/beats](https://github.com/elastic/beats) and change Filebeat @timestamp precision to nanoseconds, but i know it not the best way to solved it.

> <https://github.com/pytimer/beats/commit/8996d4f0e27415eb7d390ccc0da694bad76969bf>

Now i want to know what progress of this problem. Thanks. 🙂

---

<div class="post-metadata">

**Author:** ![smamontov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smamontov/32/32588_2.png) [@smamontov](https://discuss.elastic.co/u/smamontov)\
**Post date:** [July 10, 2018, 7:13am UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/6 "2018-07-10T07:13:32Z")

</div>

Hi! Thanks for contribution! Let's open a pull request. Though elasticsearch doesn't support it, this feature will allow us to store precise timestamp in some other text field.

---

<div class="post-metadata">

**Author:** ![pytimer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pytimer/32/29331_2.png) [@pytimer](https://discuss.elastic.co/u/pytimer)\
**Post date:** [July 10, 2018, 2:56pm UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/7 "2018-07-10T14:56:58Z")

</div>

I create a issue to [https://github.com/elastic/beats/issues/7559](https://github.com/elastic/beats/issues/7559), i am not sure if i am explain it clearly. If you have other supplements, welcome. 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 7, 2018, 2:57pm UTC](https://discuss.elastic.co/t/low-timestamp-precision-cause-wrong-order-of-events-with-docker-input/137051/8 "2018-08-07T14:57:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
