Is it possible to compress output from the S3 plugin?


(Sam Barham) #1

I'm looking at using the S3 output plugin to archive all our logs to an S3 bucket. My current system is using the file output plugin to write gzipped files to disk, which are then uploaded to S3 via a custom script. The issue I have is that the S3 plugin files are 25 times larger than the gzipped file plugin files! I don't really want to cause a a 2500% increase in our log archiving costs!

So, is there any way to make the S3 output plugin compress it's output files?


(Mark Walkom) #2

Not according to https://www.elastic.co/guide/en/logstash/current/plugins-outputs-s3.html


(system) #3