Logstash s3 output doesn't clean up temporary files

logstash version: docker 5.1.1
s3 output plugin version: 4.0.3
plugin config:

output {
  s3 {
    bucket => "some_bucket"
    access_key_id => "some_key"
    secret_access_key => "some_secret"
    region => "eu-west-1"
    codec => "json_lines"
    encoding => "none"
    size_file => "16777216"
    time_file => "1"
    upload_workers_count => "8"
    restore => "true"
  }
}

When trying to migrate from an old combination of logstash 2.4.1 + s3 output plugin 3.2.0 to the above described latest versions, the output plugin now seems to fail to clean up after itself. The temporary directories are created inside /tmp/logstash with a file inside them like so:

/tmp/logstash/02880db8-7da0-40e3-a64a-e7899d06153f/ls.s3.49146331-8fc2-48a7-8ef8-5a9854f444fc.2017-01-12T12.23.part27.txt

The files are uploaded to s3, but they remain on disk eventually filling it up.
An excerpt from the logs:

$ cat stdout | grep c79e45ffdc7.2017-01-12T12.41.part59
12:41:50.190 [[main]>worker1] DEBUG logstash.outputs.s3 - Rotate file {:strategy=>"LogStash::Outputs::S3::SizeAndTimeRotationPolicy", :key=>"ls.s3.ca3c5216-d807-4386-8c40-fc79e45ffdc7.2017-01-12T12.41.part59.txt", :path=>"/tmp/logstash/30818e0b-a354-4ab0-be56-8b8e9c29645c/ls.s3.ca3c5216-d807-4386-8c40-fc79e45ffdc7.2017-01-12T12.41.part59.txt"}
12:41:50.190 [[main]>worker1] DEBUG logstash.outputs.s3 - Queue for upload {:path=>"/tmp/logstash/30818e0b-a354-4ab0-be56-8b8e9c29645c/ls.s3.ca3c5216-d807-4386-8c40-fc79e45ffdc7.2017-01-12T12.41.part59.txt"}
12:41:53.538 [S3 output uploader, file: /tmp/logstash/30818e0b-a354-4ab0-be56-8b8e9c29645c/ls.s3.ca3c5216-d807-4386-8c40-fc79e45ffdc7.2017-01-12T12.41.part59.txt] DEBUG logstash.outputs.s3 - Removing temporary file {:file=>"/tmp/logstash/30818e0b-a354-4ab0-be56-8b8e9c29645c/ls.s3.ca3c5216-d807-4386-8c40-fc79e45ffdc7.2017-01-12T12.41.part59.txt"}

Yet the directory and the file inside it remain on disk.

Might be worth to add, that logstash is running in docker with the volume /tmp/logstash is mapped out onto the host's /tmp/logstash. The same configuration, same setup worked flawlessly with the previous versions.

Any ideas?

Thanks,

János

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.