# S3 output: Is there a way to have every event in its own file? **URL:** \ **Category:** Logstash\ **Created:** [January 7, 2023, 2:10pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663 "2023-01-07T14:10:19Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![rickfish](https://avatars.discourse-cdn.com/v4/letter/r/e480ec/32.png) [@rickfish](https://discuss.elastic.co/u/rickfish)\ **Post date:** [January 7, 2023, 2:10pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/1 "2023-01-07T14:10:19Z")
I have an unusual requirement to put every event in a separate file in an s3 bucket. I have tried to use this: ```auto rotation_strategy => "size" size_file => 1 ``` but that is just an estimate and it still puts multiple events into the same file. I have also tried to use this: `tags => "%{[event_unique_id]}"` but that is not dynamic per-event, it just puts that string into every file name. The only way I could get it to work is to set the prefix, i.e. `prefix => "%{[event_unique_id]}"` but that creates separate subfolders for each file and it decreases performance significantly. Is there a way I can accomplish this? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 7, 2023, 5:43pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/2 "2023-01-07T17:43:03Z")
> [@rickfish](#): > > Is there a way I can accomplish this? No, there is not. The [multi\_receive\_encoded](https://github.com/logstash-plugins/logstash-output-s3/blob/f893dae88ee60c2edb67608ab6a0f42f34a17235/lib/logstash/outputs/s3.rb#L243) method in the filter receives a batch of events. It processes them all and then calls the rotate\_if\_needed function. It [deliberately](https://github.com/logstash-plugins/logstash-output-s3/blob/f893dae88ee60c2edb67608ab6a0f42f34a17235/lib/logstash/outputs/s3.rb#L260) only calls fstat once for the entire batch as an optimisation. ---
**Author:** ![rickfish](https://avatars.discourse-cdn.com/v4/letter/r/e480ec/32.png) [@rickfish](https://discuss.elastic.co/u/rickfish)\ **Post date:** [January 7, 2023, 5:59pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/3 "2023-01-07T17:59:40Z")
Ok, thanks for the quick response ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 7, 2023, 6:15pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/4 "2023-01-07T18:15:45Z")
Come to think of it, if you set pipeline.batch.size to 1 then you might get the result you want. However tiny batches are less efficient that regular batches (125 events). Not sure how much less. ---
**Author:** ![rickfish](https://avatars.discourse-cdn.com/v4/letter/r/e480ec/32.png) [@rickfish](https://discuss.elastic.co/u/rickfish)\ **Post date:** [January 7, 2023, 6:57pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/5 "2023-01-07T18:57:43Z")
That works @Badger but, as you suggested, performance is really bad as there could be tons of these events. I actually question the validity of this requirement and will go back to the users and try to change it. Thanks so much for giving me the solution! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 4, 2023, 6:58pm UTC](https://discuss.elastic.co/t/s3-output-is-there-a-way-to-have-every-event-in-its-own-file/322663/6 "2023-02-04T18:58:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.