Can Logstash pull gzipped files from s3

(red der) #1

I have gzipped files on S3 I want to push into ES.

there is an S3 input plugin:

There is a gzip input plugin:

Is there a way to chain input plugins together? So the S3 input copies it locally and the gzip input unzips, groks, and pushes into ES?

(Sjaak) #2

No need, the s3 input can handle .gz files.

(red der) #3

oh wow that awesome. When it says "are handled as gzip’ed files"- this means i don't have to do anything special right? I can treat my gzipped files like regular files and grok the contents and the plugin unzips them for me transparently?

(Sjaak) #4

You don't have to do anything, Logstash handles it all for you. I'm doing the same thing with .gz logs pulled from my S3.

(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.