# S3 output can't find files it generates

**URL:** <https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546>\
**Category:** Logstash\
**Created:** [December 9, 2016, 10:44am UTC](https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546 "2016-12-09T10:44:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vad/32/3901_2.png) [@vad](https://discuss.elastic.co/u/vad)\
**Post date:** [December 9, 2016, 10:44am UTC](https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546/1 "2016-12-09T10:44:09Z")

</div>

Hi, I'm using Logstash 5.0.2 (and .0.1 before) and I need to add an s3 output but I can't because randomly one of the file logstash generates cannot be found:

```
[2016-12-07T14:37:39,809][ERROR][logstash.outputs.s3] failed to upload, will re-enqueue /tmp/logstash/ls.s3.ip-172-31-45-248.2016-12-07T14.31.part0.txt for upload {:ex=>#<Errno::ENOENT: No such file or directory - /tmp/logstash/ls.s3.ip-172-31-45-248.2016-12-07T14.31.part0.txt>, :backtrace=>["org/jruby/RubyFile.java:370:in `initialize'", "org/jruby/RubyIO.java:1197:in `open'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:175:in `write_on_bucket'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:289:in `move_file_to_bucket'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:454:in `upload_worker'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-s3-3.2.0/lib/logstash/outputs/s3.rb:436:in `configure_upload_workers'", "org/jruby/RubyProc.java:281:in `call'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24:in `initialize'"]}

```

As it says Logstash "will re-enqueue" the file over and over again filling up the disk with log messages. This happens randomly: Logstash works for some time (minutes or hours) and then suddenly this happens.

This is my output configuration:

```
if [type] == "my-log" {
  s3 {
    bucket => "mybucket"
    prefix => "logstash/bla/"
    size_file => 10485760 # rotate every 10M
    codec => line {
      format => "%{message}"
    }
  }
}

```

I'm on ubuntu 14.04. Oracle JVM:

```
# java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

```

Hints?

---

<div class="post-metadata">

**Author:** ![Hush077](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hush077/32/22500_2.png) [@Hush077](https://discuss.elastic.co/u/Hush077)\
**Post date:** [December 19, 2016, 6:52pm UTC](https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546/2 "2016-12-19T18:52:20Z")

</div>

Happening to me as well on 5.1.1.

Fills up up 20gb in less than an hour, about to have to disable it.

Also note this in a 3 node logstash cluster.

---

<div class="post-metadata">

**Author:** ![Hush077](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hush077/32/22500_2.png) [@Hush077](https://discuss.elastic.co/u/Hush077)\
**Post date:** [December 19, 2016, 7:00pm UTC](https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546/3 "2016-12-19T19:00:50Z")

</div>

Fixed mine by removing the size\_file / time\_file options.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 16, 2017, 7:00pm UTC](https://discuss.elastic.co/t/s3-output-cant-find-files-it-generates/68546/4 "2017-01-16T19:00:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.