How to parse .gz (compressed file) using logstash

[jotsnadasari]

Hi All,

Does latest logstash support parsing .gz files using logstash?

We have a requirement to get .gz files dynamically every hour. We do not know the file names and content. All we know is the file is compressed in .gz format.

We are using logstash version 5.6.0 and using gzip_lines codec plugin but logstash is not able to parse .gz files. Also using grok filter for parsing/extracting fields.

Sample code

input {
file {
type => "test-xyz"
path => "/tmp/*.gz"
start_position => "beginning"
sincedb_path => "/dev/null"
codec => "gzip_lines"
}
}

I have seen some old threads saying logstash does not support .gz files.
Wanted to check whether there is any work around for dynamic .gz files we receive.

Please provide your inputs on this.

Thanks and Regards,

[nikhilsangvikar (Nikhil S)]

No you can't. You will have to do that manually.

Here: magnusbaeck's reply on Can Logstash process archived data in the .tar.gz format?

[jotsnadasari]

Thanks Nikhil for the info.