Hi,
I’ve started testing what Logstash (7.3.0) can do and I’m having a problem with the File input plugin‘s “read mode.” It fails on gzip'd files. The problem has been reported before, e.g. here:
Apparently, the read mode part of the plugin is not compatible with Java >8. That user solved the problem by installing Java 8. My problem is, I am running Elastic’s official logstash-oss docker image, which means I cannot downgrade the JVM (unless I build a custom image based on the official one).
Does anyone have an idea or have you encountered this before? Would GitHub - elastic/logstash: Logstash - transport and process your logs, events, or other data be the appropriate repository to file a bug report?
Thanks,
Markus
For the record, this is Logstash’s output:
[2019-08-08T18:00:54,299][ERROR][logstash.javapipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::File mode=>"read", path=>["/var/log/apache2/*-error.log.*.gz"], id=>"d2db8989a7052a6e876386ccf232bb968ab0d5eee5316e27d803f9246d2ca30b", type=>"apache_error", file_completed_action=>"log", file_completed_log_path=>"/dev/null", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_179c5f0a-3fbe-40ce-813f-63d420890a2c", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, start_position=>"end", delimiter=>"\n", close_older=>3600.0, sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
Error: wrong number of arguments (given 1, expected 0)
Exception: ArgumentError
Stack:
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/read_zip_file.rb:26:in `handle_specifically'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/base.rb:26:in `handle'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:39:in `read_zip_file'
(rest of stack trace removed)
And the relevant part of my pipeline configuration looks like this:
file {
path => ["/var/log/apache2/*-error.log.*.gz"]
type => "apache_error"
mode => "read"
file_completed_action => "log"
file_completed_log_path => "/dev/null"
}