Logstash: File Input in Read Mode does not work - Docker image and Java Version


I’ve started testing what Logstash (7.3.0) can do and I’m having a problem with the File input plugin‘s “read mode.” It fails on gzip'd files. The problem has been reported before, e.g. here:

Apparently, the read mode part of the plugin is not compatible with Java >8. That user solved the problem by installing Java 8. My problem is, I am running Elastic’s official logstash-oss docker image, which means I cannot downgrade the JVM (unless I build a custom image based on the official one).

Does anyone have an idea or have you encountered this before? Would https://github.com/elastic/logstash/ be the appropriate repository to file a bug report?



For the record, this is Logstash’s output:

[2019-08-08T18:00:54,299][ERROR][logstash.javapipeline    ] A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::File mode=>"read", path=>["/var/log/apache2/*-error.log.*.gz"], id=>"d2db8989a7052a6e876386ccf232bb968ab0d5eee5316e27d803f9246d2ca30b", type=>"apache_error", file_completed_action=>"log", file_completed_log_path=>"/dev/null", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_179c5f0a-3fbe-40ce-813f-63d420890a2c", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, discover_interval=>15, sincedb_write_interval=>15.0, start_position=>"end", delimiter=>"\n", close_older=>3600.0, sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc">
  Error: wrong number of arguments (given 1, expected 0)
  Exception: ArgumentError
    /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/read_zip_file.rb:26:in `handle_specifically'
    /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/handlers/base.rb:26:in `handle'
    /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.10/lib/filewatch/read_mode/processor.rb:39:in `read_zip_file'
    (rest of stack trace removed)

And the relevant part of my pipeline configuration looks like this:

file {
    path                        => ["/var/log/apache2/*-error.log.*.gz"]
    type                        => "apache_error"
    mode                        => "read"
    file_completed_action       => "log"
    file_completed_log_path     => "/dev/null"

There is already an issue for this.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.