# File output filter writing logs out of order

**URL:** <https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654>\
**Category:** Logstash\
**Created:** [June 1, 2015, 3:26pm UTC](https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654 "2015-06-01T15:26:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brandon\_Metcalf](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@Brandon\_Metcalf](https://discuss.elastic.co/u/Brandon_Metcalf)\
**Post date:** [June 1, 2015, 3:26pm UTC](https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654/1 "2015-06-01T15:26:59Z")

</div>

This is a continuation from logstash-users; moving here.

On Mon, Jun 1, 2015 at 9:50 AM, Magnus Bäck [magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) wrote:  
On Monday, June 01, 2015 at 16:38 CEST,  
[brandon.metcalf@logicmonitor.com](mailto:brandon.metcalf@logicmonitor.com) wrote:

> > We are using pretty trivial logstash setup. Sender configs look  
> > something like

[...]

> > output {  
> > tcp {  
> > host =\> "[log02.us-east-1.logicmonitor.net](http://log02.us-east-1.logicmonitor.net)"  
> > port =\> 2009  
> > mode =\> "client"  
> > codec =\> "json"  
> > workers =\> 10  
> > }  
> > }

[...]

> > output {  
> > file {  
> > path =\> "/log/%{new-host}/%{path}"  
> > codec =\> "json"  
> > message\_format =\> "%{message}"  
> > workers =\> 30  
> > }  
> > }  
> > We are finding that logs entries are being written out of order  
> > (seemingly random) to the destination files on the receiver.

> With multiple output workers for both the tcp and file outputs that's  
> expected. There are no ordering guarantees.

I tested with just one tcp output worker and get the same behavior. I did not change the file output workers. Anything else to look at? Thanks.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 1, 2015, 4:57pm UTC](https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654/2 "2015-06-01T16:57:40Z")

</div>

I wouldn't expect there to be any ordering guarantees for multithreaded file output workers either.

---

<div class="post-metadata">

**Author:** ![Brandon\_Metcalf](https://avatars.discourse-cdn.com/v4/letter/b/8e7dd6/32.png) [@Brandon\_Metcalf](https://discuss.elastic.co/u/Brandon_Metcalf)\
**Post date:** [June 1, 2015, 7:11pm UTC](https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654/3 "2015-06-01T19:11:40Z")

</div>

Ok, thanks for additional reply. Maybe I should take a step back and describe our use case. We simply need a mechanism by which individuals that do not have access to our production systems (and therefore direct access to logs) can tail logs in real time. I increased the workers on our receivers in order to handle the large number of logs coming in which, apparently, has introduced the issue at hand.

Any suggestions on how to best solve this with logstash? We don't index into elasticsearch or anything more advanced because of our basic use case. We do feed our logs into another service for more advanced queries, but some folks like working with and viewing logs in real time.

Thanks again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:38am UTC](https://discuss.elastic.co/t/file-output-filter-writing-logs-out-of-order/1654/4 "2017-07-06T05:38:52Z")

</div>