# How to make filebeat ship logs in the same order as in the log file

**URL:** https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402
**Category:** Beats
**Tags:** filebeat
**Created:** [June 14, 2017, 2:56pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402 "2017-06-14T14:56:05Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![pratheek\_k](https://avatars.discourse-cdn.com/v4/letter/p/dc4da7/32.png) [@pratheek\_k](https://discuss.elastic.co/u/pratheek_k)
#### Post date: [June 14, 2017, 2:56pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402/1 "2017-06-14T14:56:05Z")

</div>

I can see that filebeat is not shipping the logs in the same order to logstash as in the log file. So can i make filebeat to ship the logs in the same order as in the log file. Sorting it later in elasticsearch based on timestamp does not solve my need because i would like to make sure that my log for request from user is ordered before combining into a single event in logstash based on unique id.

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [June 14, 2017, 11:44pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402/2 "2017-06-14T23:44:25Z")

</div>

Assuming you are outputting to single LS instance, using a single worker, and have async publishing disabled, I would expect the delivery of log messages from a single file to be in order (possibly interleaved with message from other log files being read). Could it happening on the Logstash side? Based on the [execution model](https://www.elastic.co/guide/en/logstash/current/execution-model.html) described on that page, I think it could.

In general Beats does not provide order delivery guarantees, only an at-least once delivery guarantee.

Would it be possible to use multiline on the Beats side to combine the related log lines into a single event before sending it to Logstash? Then once at logstash you could parse all the data from one event. This would require that the related log lines are not interleaved with other log messages.

---

<div class="post-metadata">

### Author: ![pratheek\_k](https://avatars.discourse-cdn.com/v4/letter/p/dc4da7/32.png) [@pratheek\_k](https://discuss.elastic.co/u/pratheek_k)
#### Post date: [June 15, 2017, 3:25pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402/3 "2017-06-15T15:25:35Z")

</div>

@andrewkroh

I was researching and found I dont think this cannot be done by filebeat. Please see this post [Filebeat multiline by Queue ID](https://discuss.elastic.co/t/filebeat-multiline-by-queue-id/47406)

Also, i found the below in the documentation,[https://www.elastic.co/guide/en/logstash/current/multiline.html](https://www.elastic.co/guide/en/logstash/current/multiline.html)

If you are using a Logstash input plugin that supports multiple hosts, such as the beats input plugin, you should not use the multiline codec to handle multiline events. Doing so may result in the mixing of streams and corrupted event data. In this situation, you need to handle multiline events before sending the event data to Logstash.

If you think it logstash might pick up threads in disorder how can i make in order.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 13, 2017, 3:25pm UTC](https://discuss.elastic.co/t/how-to-make-filebeat-ship-logs-in-the-same-order-as-in-the-log-file/89402/4 "2017-07-13T15:25:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
