Filewatch.tailmode.handlers.createinitial open_file OPEN_WARN_INTERVAL is 300

Elastic Stack Logstash
1

Hi, Logstash has started outputting the following message sometimes:

[2018-07-02T**10:19**:11,155][WARN ][filewatch.tailmode.handlers.createinitial] open_file OPEN_WARN_INTERVAL is '300'                         
...
[2018-07-02T10:29:12,677][WARN ][filewatch.tailmode.handlers.createinitial] failed to open /ssd/staging/etc/messages-20180627: #<Errno::EACCES: Permission denied - /ssd/staging/logs/etc/messages-20180627>, ["org/jruby/RubyFile.java:366:in `initialize'", "org/jruby/RubyIO.java:1154:in `open'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-file-4.1.3/lib/filewatch/watched_file.rb:73:in `open'"]                                                                                                                                   
...
[2018-07-02T**10:30**:03,804][WARN ][filewatch.tailmode.handlers.createinitial] open_file OPEN_WARN_INTERVAL is '300'

I have corrected the permissions on the file which failed, but the fact that message appeared after hundreds of the OPEN_WARN_INTERVAL tells me that it is spending all that time reading the files in that input

I suspect it is caused by the number of files I have listed in my file input:

> /etc/logstash/conf.d/inputs.conf
file {
    path => ["/ssd/staging/**/**/**/*", "/ssd/staging/**/**/*"]
    start_position => "beginning"
    exclude => ["*.gz", "*.xz"]
  }

> find /ssd/staging/ | wc -l
1681

Is there a setting to either suppress the message, increase the open file limit of the filewatch.tailmode program, or is the file input just not intended to watch so many files?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.