Can Logstash pull gzipped files from s3

Elastic Stack Logstash
1

I have gzipped files on S3 I want to push into ES.

there is an S3 input plugin: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html

There is a gzip input plugin: https://www.elastic.co/guide/en/logstash/current/plugins-codecs-gzip_lines.html

Is there a way to chain input plugins together? So the S3 input copies it locally and the gzip input unzips, groks, and pushes into ES?

2

No need, the s3 input can handle .gz files.

3

oh wow that awesome. When it says "are handled as gzip’ed files"- this means i don't have to do anything special right? I can treat my gzipped files like regular files and grok the contents and the plugin unzips them for me transparently?

4

You don't have to do anything, Logstash handles it all for you. I'm doing the same thing with .gz logs pulled from my S3.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.