Can Logstash pull gzipped files from s3

I have gzipped files on S3 I want to push into ES.

there is an S3 input plugin: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html

There is a gzip input plugin: https://www.elastic.co/guide/en/logstash/current/plugins-codecs-gzip_lines.html

Is there a way to chain input plugins together? So the S3 input copies it locally and the gzip input unzips, groks, and pushes into ES?

No need, the s3 input can handle .gz files.

oh wow that awesome. When it says "are handled as gzip’ed files"- this means i don't have to do anything special right? I can treat my gzipped files like regular files and grok the contents and the plugin unzips them for me transparently?

You don't have to do anything, Logstash handles it all for you. I'm doing the same thing with .gz logs pulled from my S3.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.