Logstash I/O error

[Hardy_Fong]

Dear all ,

I have one elk server collecting all decive syslog, and we found one of fibre switch(DELL DS6610B) have prompt this error in starting logstash.

But the weird point is we have 3 fibre switch which are same config and permisson

another 2 fibre switch are ok, just only this fibre switch prompting error

ERROR log

/elklog/logstash-7.6.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.16/lib/filewatch/sincedb_collection.rb:22:in `initialize'
/elklog/logstash-7.6.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.16/lib/filewatch/observing_base.rb:62:in `build_watch_and_dependencies'
/elklog/logstash-7.6.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.16/lib/filewatch/observing_base.rb:56:in `initialize'
/elklog/logstash-7.6.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.16/lib/logstash/inputs/file.rb:341:in `start_processing'
/elklog/logstash-7.6.1/vendor/bundle/jruby/2.5.0/gems/logstash-input-file-4.1.16/lib/logstash/inputs/file.rb:346:in `run'
/elklog/logstash-7.6.1/logstash-core/lib/logstash/java_pipeline.rb:328:in `inputworker'
/elklog/logstash-7.6.1/logstash-core/lib/logstash/java_pipeline.rb:320:in `block in start_input'
[2022-09-07T15:39:43,836][INFO ][filewatch.observingtail  ][main] START, creating Discoverer, Watch with file and sincedb collections
[2022-09-07T15:39:43,840][ERROR][logstash.javapipeline    ][main] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:main
  Plugin: <LogStash::Inputs::File start_position=>"beginning", path=>["/elklog/10.200.11.61/*.log"], discover_interval=>10, id=>"dd1fa9797af5e0adb8183cd5d53267c79d79cb63c6205c1b24cb698f941a39e8", type=>"200_11_61_log", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_b11be65d-9a38-48ac-bf2a-e36d1c44a11c", enable_metric=>true, charset=>"UTF-8">, stat_interval=>1.0, sincedb_write_interval=>15.0, delimiter=>"\n", close_older=>3600.0, mode=>"tail", file_completed_action=>"delete", sincedb_clean_after=>1209600.0, file_chunk_size=>32768, file_chunk_count=>140737488355327, file_sort_by=>"last_modified", file_sort_direction=>"asc", exit_after_read=>false>
  Error: Input/output error
  Exception: IOError
  Stack: org/jruby/RubyIO.java:1236:in `sysopen'
org/jruby/RubyFile.java:367:in `initialize'
org/jruby/RubyIO.java:1158:in `open'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1136:in `block in touch'
org/jruby/RubyArray.java:1814:in `each'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/fileutils.rb:1130:in `touch'

Config of problem fibre switch

......

input {
  file{
    path => ["/elklog/10.200.11.61/*.log"]
    type => "200_11_61_log"
    discover_interval => 10
    start_position => "beginning"
  }
}

.....

  if[type] == "200_11_61_log" {
   elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "200_11_61_log-%{+YYYY.MM.dd}"
  }
  }

.....

another Config of fibre switch
.....

input {
  file{
    path => ["/elklog/10.100.11.62/*.log"]
    type => "100_11_62_log"
    discover_interval => 10
    start_position => "beginning"
  }
}

.......

  if[type] == "100_11_61_log" {
   elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "100_11_61_log-%{+YYYY.MM.dd}"
  }
  }

......