# Does Logstash (7.0.1) support gzip file input? **URL:** https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862 **Category:** Logstash **Created:** [May 20, 2019, 3:22pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862 "2019-05-20T15:22:35Z") **Posts on this page:** 8 **Page:** 1
### Author: ![nobound](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@nobound](https://discuss.elastic.co/u/nobound) #### Post date: [May 20, 2019, 3:22pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/1 "2019-05-20T15:22:35Z")
Hi all, It seems there are issues regarding gzip file support, but I can't really find any post that will give me the answer. I have tried to use gzip\_lines codec. It does not work. ``` input { file { path => "/path/*.gz" codec => "gzip_lines" } } output { stdout { codec => rubydebug } } ``` The file input plugin ([https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html)) seems to indicate the "read" mode now supports gzip file processing. ``` input { file { path => "/path/test.log.gz" mode => "read" file_completed_action => "log" file_completed_log_path => "/path/output_completed_log.log" } } output { stdout { codec => rubydebug } } ``` With "mode" set to read, I am getting argument error. ``` [2019-05-20T09:45:33,403][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=>"main"} [2019-05-20T09:45:33,449][INFO][filewatch.observingread] START, creating Discoverer, Watch with file and sincedb collections [2019-05-20T09:45:33,454][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} [2019-05-20T09:45:33,665][ERROR][logstash.javapipeline] A plugin had an unrecoverable error. Will restart this plugin. Pipeline_id:main Plugin: "read", path=>["/Users/kwlee/workspace/purge/data/test.log.gz"], discover_interval=>40, file_sort_by=>"path", id=>"18d120c5852688eeea2e022776fe4faaf72378c29c18a692315259782ec16260", file_completed_action=>"log", file_completed_log_path=>"/Users/kwlee/workspace/purge/data/gz-local-read-file-mode-completed.log", stat_interval=>0.1, enable_metric=>true, codec=>"plain_b6fdea91-b564-436b-ac87-7020bff4afbb", enable_metric=>true, charset=>"UTF-8">, sincedb_write_interval=>15.0, start_position=>"end", delimiter=>"\n", close_older=>3600.0, sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_direction=>"asc"> Error: wrong number of arguments (given 1, expected 0) Exception: ArgumentError Stack: /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/read_zip_file.rb:26:in `handle_specifically' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/base.rb:26:in `handle' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:39:in `read_zip_file' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:102:in `block in process_active' org/jruby/RubyArray.java:1792:in `each' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:88:in `process_active' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:45:in `process_all_states' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/watch.rb:67:in `iterate_on_state' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/watch.rb:45:in `subscribe' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/observing_read.rb:12:in `subscribe' /Users/kwlee/workspace/purge/logstash-7.0.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/logstash/inputs/file.rb:339:in `run' /Users/kwlee/workspace/purge/logstash-7.0.1/logstash-core/lib/logstash/java_pipeline.rb:297:in `inputworker' /Users/kwlee/workspace/purge/logstash-7.0.1/logstash-core/lib/logstash/java_pipeline.rb:290:in `block in start_input' ``` Without "mode" set to read, I get unreadable message ``` "message" => "$\\xED\\x87S\\xF4h\\xC8״\\xA9!E!\\xA7\\x94'\\xE5\\x8C\\u0005m\\x99\\x8Cs\\u007F\\xCCU\\u0006\\x86H\\u001AS\\xFB\\xEE\\u0018\\x96\\u0006\\x9C\\u0002\\x96\\x82~M4(\\xC0\\x91`\\u001A\\x9E\\xA5\\u0000\\xB0\\xCB|\\x96&\\xC14\\xA2XJ\\x82\\xD3\\xE0=8\\r\\xB5\\xEE\\xC9$\\b\\xA7\\xE1\\xC5!L\\xA4\\xDB\\xE8dȳӨ\\xB6\\u00038\\r\\xB3\\u0016\\\\\\x83\\u0006\\u000FZ\\xAF9\\u0005N\\xC33'\\x97\\xD3eQ8\\x8D\\x802\\xDBAf|{\\t/&\\xE4\\xB2Q\\xDA/\\xE9m\\xF3$C\\u001E\\xF5X\\u0018\\x94\\x9A\\u0010\\u0016\\x86\\xE7_Ǻ\\u000EN\\u0016@\\b\\v\\x83+\\xB3\\xA9\\xEB^H\\xC2VԬ\\xC0\\x8E\\xDBn%ۀ\\xFD'B\\xE9\\xF5\\x9E\\u0001\\xB3\\u0018\\u0003\\x8B\\x875\\r\\xA1x09\\x99\\xF8g\\rk\\u001A\\\"\\xF0\\x9E\\u001D9\\xD9I\\u0003\\u0003\\x8B\\xEA\\x83zBX\\xD3\\xC1\\u0012\\u0004Cc3YO\\xC7\\xC2\\b\\xEF\\xB4\\u0018\\u000E5\\x9Es\\x94bK&aa\\xE4\\xE0\\xCER\\xB4\\xF3\\xA0\\u0012\\b\\u0018\\xB1\\xA2\\xF9",` ``` I really appreciate If someone can point me in the right direction. Thanks ---
### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger) #### Post date: [May 20, 2019, 4:04pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/2 "2019-05-20T16:04:18Z")
That's odd. The line that is throwing an exception is [this](https://github.com/logstash-plugins/logstash-input-file/blob/082897a66c8410b6b343083146307275d530c88b/lib/filewatch/read_mode/handlers/read_zip_file.rb#L26). The readline method of a java.io.BufferedReader does not take an argument, so I would expect that exception. However, it works just fine for me. ---
### Author: ![nobound](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@nobound](https://discuss.elastic.co/u/nobound) #### Post date: [May 20, 2019, 4:42pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/3 "2019-05-20T16:42:57Z")
Thanks for the response. Your logstash config file is somewhat similar to following, and it works for you. Is that right? ``` input { file { path => "/path/test.log.gz" mode => "read" file_completed_action => "log" file_completed_log_path => "/path/output_completed_log.log" } } output { stdout { codec => rubydebug } } ``` I download the logstash (tar.gz file) from [https://www.elastic.co/downloads/logstash](https://www.elastic.co/downloads/logstash), and I am currently running the logstash on mac. ``` # ruby --version ruby 2.3.7p456 (2018-03-28 revision 63024) [universal.x86_64-darwin18] # java --version openjdk 12.0.1 2019-04-16 OpenJDK Runtime Environment (build 12.0.1+12) OpenJDK 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing) ``` Do you have similar setup environment? Thanks ---
### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger) #### Post date: [May 20, 2019, 4:47pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/4 "2019-05-20T16:47:37Z")
> [@nobound](#): > > Your logstash config file is somewhat similar to following, and it works for you. Is that right? Correct. The paths are different but the setup is the same. jruby 9.2.7.0 (2.5.3) OpenJDK 64-Bit Server VM 25.201-b09 on 1.8.0\_201-b09 ---
### Author: ![nobound](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@nobound](https://discuss.elastic.co/u/nobound) #### Post date: [May 20, 2019, 6:05pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/5 "2019-05-20T18:05:52Z")
The java version on my mac could be the cause of the issue. I try the same lagstash config file on the Linux machine with java 8, and it works. ``` $ java -version openjdk version "1.8.0_191" OpenJDK Runtime Environment (build 1.8.0_191-8u191-b12-2ubuntu0.16.04.1-b12) OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) ``` I did not realize this could be an issue since everything else worked. I will try to install java 8 on my mac to see if it fixes the issue. Thanks again for your help. ---
### Author: ![nobound](https://avatars.discourse-cdn.com/v4/letter/n/f07891/32.png) [@nobound](https://discuss.elastic.co/u/nobound) #### Post date: [May 23, 2019, 5:35pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/6 "2019-05-23T17:35:37Z")
Hi I have installed java 8 on my mac, and it seems it is working. I am just wondering if I should open a ticket to track this? Thanks Kc ---
### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger) #### Post date: [May 23, 2019, 5:49pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/7 "2019-05-23T17:49:01Z")
> [@nobound](#): > > I am just wondering if I should open a ticket to track this? I think so, although Java 12 is not on the [support matrix](https://www.elastic.co/support/matrix#matrix_jvm) for logstash. ---
### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system) #### Post date: [June 20, 2019, 5:49pm UTC](https://discuss.elastic.co/t/does-logstash-7-0-1-support-gzip-file-input/181862/8 "2019-06-20T17:49:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.