# Ensuring order for syslog events

**URL:** <https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 6, 2019, 1:07pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117 "2019-03-06T13:07:07Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![thro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thro/32/29022_2.png) [@thro](https://discuss.elastic.co/u/thro)\
**Post date:** [March 6, 2019, 1:07pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/1 "2019-03-06T13:07:07Z")

</div>

Hello,

I have a problem trying with ordering of syslog events once they enter ElasticSearch.

The problem is that the syslog daemon only has a resolution to the second, so the file itself is in correct order but there is no way to order them correctly in Elasticsearch.

Is there any way to send somekind of tiebreaker to ensure order using filebeat, even if it is the line number?

.thro

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [March 6, 2019, 2:51pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/2 "2019-03-06T14:51:58Z")

</div>

This is a good question, maybe a workaround would be to use the date processor in the ingest node to create a new date based on the extracted date which has the second resolution and use the offset or part of the offset as the nanosecond resolution?

---

<div class="post-metadata">

**Author:** ![thro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thro/32/29022_2.png) [@thro](https://discuss.elastic.co/u/thro)\
**Post date:** [March 7, 2019, 10:06am UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/3 "2019-03-07T10:06:36Z")

</div>

Yes, had that idea. But I read somewhere that you have to make sure that Filebeat is only running with one thread to ensure that it will be sent in the right order, and I can't guarantee that nobody will change the clients in the future or any other relevant settings along the way.

If Filebeat could parse a custom key (date with a res. of a second and no year in my case) and add tiebreaker number with order of appearance then I'd say that we had a lossless solution on our hands. At least content wise.

Which begs the question, is it possible to write a custom parser for a line before it is sent?

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [March 7, 2019, 1:45pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/4 "2019-03-07T13:45:56Z")

</div>

Concerning the one thread even with that, a network or multiple workers could affect the ordering or events. I wonder if using LS directly might be the solution here.

---

<div class="post-metadata">

**Author:** ![thro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thro/32/29022_2.png) [@thro](https://discuss.elastic.co/u/thro)\
**Post date:** [March 7, 2019, 2:13pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/5 "2019-03-07T14:13:05Z")

</div>

Well Logstash is handling parsing the timestamp correctly for now, but I think the safest choice is to do it on the device itself.

Does Logstash have any notion for the concept of past events?

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [March 7, 2019, 2:50pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/6 "2019-03-07T14:50:00Z")

</div>

I gave a bit more thinking about and searched a bit about what other did and I've seen this [answer](https://discuss.elastic.co/t/restore-the-sequence-of-the-events/67766/) from a colleague, it might be the solution here.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2019, 2:50pm UTC](https://discuss.elastic.co/t/ensuring-order-for-syslog-events/171117/7 "2019-04-04T14:50:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
