# Load logs from gzip files using codec

**URL:** <https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188>\
**Category:** Logstash\
**Created:** [October 5, 2018, 10:06am UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188 "2018-10-05T10:06:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![GitSpree23](https://avatars.discourse-cdn.com/v4/letter/g/65b543/32.png) [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\
**Post date:** [October 5, 2018, 10:06am UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188/1 "2018-10-05T10:06:10Z")

</div>

There is a closed topic regarding the same from 2 years ago ...

> [@Read a gzip file with gzip\_lines codec](https://discuss.elastic.co/t/read-a-gzip-file-with-gzip-lines-codec/24558/12):
>
> Clone Source git clone https://github.com/logstash-plugins/logstash-codec-gzip\_lines Change gzip\_lines.rb logstash-codec-gzip\_lines/lib/logstash/codecs/gzip\_lines.rb line 32 @decoder = Zlib::GzipReader.open(data) Build Gem File cd logstash-codec-gzip\_lines gem build logstash-code-gzip\_lines.gemspec Reinstall logstash-codec-gzip\_lines cd logstash bin/plugin install /opt/logstash-codec-gzip\_lines/logstash-codec-gzip\_lines-2.0.2.gem Create list.txt File Write by yourself gzip\_line…

But the .rb code has been updated on GitHub.

What is the latest solution to using the gzip-lines codec?

I have installed the codec and am running the following conf ...

```
input {
        file {
                type => "gzip"
                path => "/home/aptusdb/data/list.txt" #path to list.txt containing path/filename of all .gz files
                start_position => "beginning"
                sincedb_path => "gzip"
                codec => "gzip_lines" #gzip_lines codec
        }
}

output {
        elasticsearch {
                hosts => "localhost"
                index => "trial1"
                document_type => "sold_cars2"
        }
        stdout {}
}

```

But I am getting the following error ...

`[2018-10-05T15:33:45,695][ERROR][filewatch.tailmode.handlers.grow] read_to_eof: general error reading /home/aptusdb/data/list.txt {"error"=>"java.lang.IllegalArgumentException: Object: data/sample/day=2018-09-23/hour=23/sample_data.txt.gz is not a legal argument to this wrapper, cause it doesn't respond to \"read\".", "backtrace"=>["org.jruby.util.IOInputStream.<init>(IOInputStream.java:64)", "org.jruby.ext.zlib.JZlibRubyGzipReader.initialize(JZlibRubyGzipReader.java:115)", "org.jruby.ext.zlib.JZlibRubyGzipReader.initialize19(JZlibRubyGzipReader.java:145)", "org.jruby.ext.zlib.JZlibRubyGzipReader$INVOKER$i$0$1$initialize19.call(JZlibRubyGzipReader$INVOKER$i$0$1$initialize19.gen)"]}`

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2018, 3:02pm UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188/2 "2018-10-05T15:02:36Z")

</div>

The file input plugin now has a [read mode](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_read_mode) that may be what you are looking for as it supports reading compressed files.

---

<div class="post-metadata">

**Author:** ![GitSpree23](https://avatars.discourse-cdn.com/v4/letter/g/65b543/32.png) [@GitSpree23](https://discuss.elastic.co/u/GitSpree23)\
**Post date:** [October 8, 2018, 4:43am UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188/3 "2018-10-08T04:43:37Z")

</div>

Thanks a lot!

Here's a reference for those facing the same issue:

```
input {
	file {
		#type => "gzip" (not compulsorily required?)
		path => "/home/user/data/**/*" #path to .gz files contained in subfolders
		#sincedb_path => "gzip" (not compulsorily required?)
		mode => "read"
		file_completed_action => "log"
		file_completed_log_path => "/home/user/data/log.txt"
	}
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2018, 4:43am UTC](https://discuss.elastic.co/t/load-logs-from-gzip-files-using-codec/151188/4 "2018-11-05T04:43:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.