# Is it possible to split a logstash file output into a random number of output files for throughput reasons?

**URL:** <https://discuss.elastic.co/t/is-it-possible-to-split-a-logstash-file-output-into-a-random-number-of-output-files-for-throughput-reasons/263909>\
**Category:** Logstash\
**Created:** [February 10, 2021, 5:21pm UTC](https://discuss.elastic.co/t/is-it-possible-to-split-a-logstash-file-output-into-a-random-number-of-output-files-for-throughput-reasons/263909 "2021-02-10T17:21:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jethropickering](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jethropickering/32/83736_2.png) [@jethropickering](https://discuss.elastic.co/u/jethropickering)\
**Post date:** [February 10, 2021, 5:21pm UTC](https://discuss.elastic.co/t/is-it-possible-to-split-a-logstash-file-output-into-a-random-number-of-output-files-for-throughput-reasons/263909/1 "2021-02-10T17:21:29Z")

</div>

Hi all.

We are writing to file at about 5000-20000 eps at times. logstash is able to do this fine. The reason we need to write to file and not an endpoint is because of a custom solution(that can't be changed) that reads the file and is bottlenecking.  
has anyone had any luck splitting a file output into multiple files? ie say you originally output to a single file called "file",is there an easy way to roundrobin to 3 seperate files instead like file1, file2, file3? The

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 10, 2021, 6:22pm UTC](https://discuss.elastic.co/t/is-it-possible-to-split-a-logstash-file-output-into-a-random-number-of-output-files-for-throughput-reasons/263909/2 "2021-02-10T18:22:24Z")

</div>

I have not tested it but you could do something like

```
filter { ruby { code => 'event.set("[@metadata][file]", Random.rand(3).to_i)' } }
output { file { path => "/some/path/file%{[@metadata][file]}" ... } }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 10, 2021, 6:22pm UTC](https://discuss.elastic.co/t/is-it-possible-to-split-a-logstash-file-output-into-a-random-number-of-output-files-for-throughput-reasons/263909/3 "2021-03-10T18:22:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.