# Logstash file output plugin writing to the same file from multiple Logstash processes?!

**URL:** <https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750>\
**Category:** Logstash\
**Created:** [January 21, 2021, 9:03am UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750 "2021-01-21T09:03:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bjosve](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@bjosve](https://discuss.elastic.co/u/bjosve)\
**Post date:** [January 21, 2021, 9:03am UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750/1 "2021-01-21T09:03:40Z")

</div>

Hi,

In the following post,

[https://discuss.elastic.co/t/multiple-logstash-nodes-use-file-output-plugin-to-output-messages-to-one-file-in-a-shared-file-system/94275/9](https://discuss.elastic.co/t/multiple-logstash-nodes-use-file-output-plugin-to-output-messages-to-one-file-in-a-shared-file-system/94275/9)

@magnusbaeck writes,

> If Logstash is opening the output file with O\_APPEND (which it should) then all write() operations will be made at the end of the file even if multiple processes write to the file concurrently.

This does not seem to be true. We have a Kafka cluster with several Logstash (7.4.1) consumers (docker containers) that write to an NFS share. When multiple Logstash instances try to write to the same file, it becomes corrupt.

So my question is. Supports Logstash file output plugin writing to the same file from different Logstash processes or not?

Thanks.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [January 21, 2021, 1:16pm UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750/2 "2021-01-21T13:16:49Z")

</div>

> [@bjosve](#):
>
> When multiple Logstash instances try to write to the same file, it becomes corrupt.

The NFS protocol does not support multiple writes to the same file, it can not guarantee that the writes will be atomic and that the file won't be corrupted.

From the [nfs faq](http://nfs.sourceforge.net)

```
A9. Why does opening files with O_APPEND on multiple clients cause the files to become corrupted?
A. The NFS protocol does not support atomic append writes, so append writes are never atomic on NFS for any platform.

```

---

<div class="post-metadata">

**Author:** ![bjosve](https://avatars.discourse-cdn.com/v4/letter/b/ecae2f/32.png) [@bjosve](https://discuss.elastic.co/u/bjosve)\
**Post date:** [February 1, 2021, 9:18am UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750/3 "2021-02-01T09:18:27Z")

</div>

Thank you. That explains a lot.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 1, 2021, 9:18am UTC](https://discuss.elastic.co/t/logstash-file-output-plugin-writing-to-the-same-file-from-multiple-logstash-processes/261750/4 "2021-03-01T09:18:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.