# Unable to read .gz files in logstash **URL:** \ **Category:** Logstash\ **Created:** [June 16, 2018, 12:43pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174 "2018-06-16T12:43:55Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [June 16, 2018, 12:43pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/1 "2018-06-16T12:43:55Z")
Hi, I am trying to read .gz files in logstash input plugin but I can't read gunzip files in logstash. When I try to read those log files it throws error message like this, **A plugin had an unrecoverable error. Will restart this plugin.** **Plugin: \["/test.txt"], start\_position=\>"beginning", sincedb\_path=\>"/dev/null", codec=\>\"UTF-8"\>, stat\_interval=\>1, discover\_interval=\>15, sincedb\_write\_interval=\>15, delimiter=\>"\n"\>** **Error: Object: application.log.2018-05-14-01.eradar-data-service-prod-1e-5af1c225.us-east-1.amazon.com.gz is not a legal argument to this wrapper, cause it doesn't respond to "read". {:level=\>:error}** LogStash.conf file: file { path =\> "/test.txt" (this file contains list of gunzip files) start\_position =\> "beginning" sincedb\_path =\> "/dev/null" codec =\> "gzip\_lines" } and also tried path =\> "/logfile.gz" directly but both are not working it throws same error message. How to solve this problem, please share your solutions. I refered all the similar questions but none of the solutions solve my issue. Thanks, ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [June 18, 2018, 11:47pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/2 "2018-06-18T23:47:43Z")
The [`path` directive](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-path) enables you to specify a glob-type _path_ to files that will be read (e.g., `application.log.*.gz`). ---
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [June 19, 2018, 4:43am UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/3 "2018-06-19T04:43:14Z")
Sorry, I don't understand your solution. can you explain some more detail? Thanks. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [June 19, 2018, 7:13pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/4 "2018-06-19T19:13:35Z")
When you specify a path that matches a file, Logstash will attempt to ingest _that_ file. Since you are giving a path to a plain-text file, and it is trying to read it with gzip, it fails. You need to give it a glob-style path that matches the files you want to ingest. ---
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [June 20, 2018, 5:48am UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/5 "2018-06-20T05:48:13Z")
I also tried that method but it does not work again same error message. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [June 21, 2018, 8:13am UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/6 "2018-06-21T08:13:23Z")
If you use the latest file input v4.1.3 you can read `gz` files directly without the GzipLines codec. Here is a simple test config: ```auto input { file { path => "/Users/guy/tmp/testing/logs/pan*.gz" sincedb_path => "/Users/guy/tmp/testing/gz-local-read-file-mode.sdb" discover_interval => 40 stat_interval => 0.1 mode => "read" file_completed_action => "log" file_completed_log_path => "/Users/guy/tmp/testing/gz-local-read-file-mode-completed.log" file_sort_by => "path" } } output { stdout { codec => dots } } ``` ---
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [June 22, 2018, 7:49am UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/7 "2018-06-22T07:49:08Z")
My Logstash version - 2.1.0 ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 22, 2018, 7:51am UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/8 "2018-06-22T07:51:51Z")
That is a very old version. I would recommend that you upgrade. ---
**Author:** ![Mormaii](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mormaii/32/10790_2.png) [@Mormaii](https://discuss.elastic.co/u/Mormaii)\ **Post date:** [July 12, 2018, 5:28pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/9 "2018-07-12T17:28:23Z")
Is there a recommended way to process around 100,000 files without having logstash die? ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [July 16, 2018, 12:27pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/10 "2018-07-16T12:27:35Z")
Not yet. I have some experimental code that "feeds" batches of discovered files to the processing stage but it is not ready yet. I hope that you are not tailing hundreds of thousands of files. ---
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [July 16, 2018, 12:29pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/11 "2018-07-16T12:29:06Z")
Hi, I can't read gzip files after updating file input plugin. Thanks, Arun ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [July 16, 2018, 12:57pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/12 "2018-07-16T12:57:28Z")
The file input at v4.1.X has test file fixtures and test that verify the code can read gz files - these fixtures, test and the actual zipfile reading code are identical to the S3 input gz reading implementation. You are experiencing a different outcome - perhaps because you have some differences in your setup. You will need to uncover what these are before anyone can help. If the LS logs show some errors or logging lines to do with `filewatch.readmode.handlers.readzipfile`then post them here. Have you tested a fresh file gzipped with `tar cvzf somefile.gz /some/pathto/somefile.x`? ---
**Author:** ![clash\_of\_clanss](https://avatars.discourse-cdn.com/v4/letter/c/da6949/32.png) [@clash\_of\_clanss](https://discuss.elastic.co/u/clash_of_clanss)\ **Post date:** [July 16, 2018, 1:11pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/13 "2018-07-16T13:11:40Z")
Hi, I am not reading logs from S3. so is it okay? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 13, 2018, 1:11pm UTC](https://discuss.elastic.co/t/unable-to-read-gz-files-in-logstash/136174/14 "2018-08-13T13:11:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.