Logstash elasticsearch input plugin observes frequent Faraday::TimeoutError. How can we increase the default timeout settings for the request?

Elastic Stack Logstash
1

We are using Logstash with an elastic search input plugin and elastic search output plugin. The data volume increased heavily during the peak hours between 10 AM - 5 PM Eastern time. And during this time frame, we observe a lot of timeouts in Logstash logs related to the input plugin.

I have checked to access the source cluster through curl API and sometimes it takes more than 1 minute (even at sometimes close to 3 minutes for response)

Is there a way to increase the timeout parameter value? can someone please guide me to resolve this issue?

//{ 2954 rufus-scheduler intercepted an error:
  2954   job:
  2954     Rufus::Scheduler::CronJob "*/5 * * * *" {}
  2954   error:
  2954     2954
  2954     Faraday::TimeoutError
  2954     Net::ReadTimeout
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/protocol.rb:181:in `rbuf_fill'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/protocol.rb:157:in `readuntil'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/protocol.rb:167:in `readline'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http/response.rb:40:in `read_status_line'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http/response.rb:29:in `read_new'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:1504:in `block in transport_request'
  2954       org/jruby/RubyKernel.java:1193:in `catch'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:1501:in `transport_request'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:1474:in `request'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:1467:in `block in request'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:914:in `start'
  2954       /opt/app/logstash/vendor/jruby/lib/ruby/stdlib/net/http.rb:1465:in `request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:87:in `perform_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:43:in `block in call'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:92:in `with_net_http_connection'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/adapter/net_http.rb:38:in `call'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:143:in `build_response'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:387:in `run_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/faraday.rb:23:in `block in perform_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/base.rb:262:in `perform_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-transport-5.0.5/lib/elasticsearch/transport/client.rb:131:in `perform_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/elasticsearch-api-5.0.5/lib/elasticsearch/api/actions/search.rb:183:in `search'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.3.2/lib/logstash/inputs/elasticsearch.rb:285:in `search_request'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.3.2/lib/logstash/inputs/elasticsearch.rb:233:in `do_run_slice'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.3.2/lib/logstash/inputs/elasticsearch.rb:214:in `do_run'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-elasticsearch-4.3.2/lib/logstash/inputs/elasticsearch.rb:197:in `block in run'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:234:in `do_call'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:258:in `do_trigger'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:300:in `block in start_work_thread'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:299:in `block in start_work_thread'
  2954       org/jruby/RubyKernel.java:1425:in `loop'
  2954       /opt/app/logstash/vendor/bundle/jruby/2.5.0/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:289:in `block in start_work_thread'
  2954   tz:
  2954     ENV['TZ']:
  2954     Time.now: 2020-10-19 09:51:00 -0400
  2954   scheduler:
  2954     object_id: 2006
  2954     opts:
  2954       {:max_work_threads=>1}
  2954       frequency: 0.3
  2954       scheduler_lock: #<Rufus::Scheduler::NullLock:0x713581bf>
  2954       trigger_lock: #<Rufus::Scheduler::NullLock:0x7368de6>
  2954     uptime: 1667335.687528 (2w5d7h8m55s687)
  2954     down?: false
  2954     threads: 2
  2954       thread: #<Thread:0x77838d8e>
  2954       thread_key: rufus_scheduler_2006
  2954       work_threads: 1
  2954         active: 1
  2954         vacant: 0
  2954         max_work_threads: 1
  2954       mutexes: {}
  2954     jobs: 1
  2954       at_jobs: 0
  2954       in_jobs: 0
  2954       every_jobs: 0
  2954       interval_jobs: 0
  2954       cron_jobs: 1
  2954     running_jobs: 1
  2954     work_queue: 0
} ```
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.