Need to restart filebeat time to time

Hi i have issue on filebeat Logs are not showing every 6hours(approximately) in Kibana dashboard. After restarting the filebeat ,Kibana dashboard gets logs again .my filebeat needs to restart around every 6 hours to keep working fine the log shipping process.

Do you see any errors in logs? Did you try to enable debug logging?

Hi @mtojek
Error in logstash server log file /var/log/logstash/logstash-plain.log

[ERROR][logstash.outputs.amazones] Attempted to send a bulk request to Elasticsearch configured at '["https://search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided? {:client_config=>{:hosts=>["https://search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443"], :region=>"us-east-1", :transport_options=>{:request=>{:open_timeout=>0, :timeout=>60}, :proxy=>nil, :headers=>{"Content-Type"=>"application/json"}}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::AWS}, :error_message=>"search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443 failed to respond", :error_class=>"Faraday::ClientError", :backtrace=>nil}

[2021-04-01T21:56:09,911][WARN ][logstash.outputs.amazones] Failed to flush outgoing items {:outgoing_count=>1, :exception=>"Faraday::ClientError", :backtrace=>nil}

[2021-04-01T22:50:07,628][ERROR][logstash.outputs.amazones] Attempted to send a bulk request to Elasticsearch configured at '["https://search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443"]', but an error occurred and it failed! Are you sure you can reach elasticsearch from this machine using the configuration provided? {:client_config=>{:hosts=>["https://search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443"], :region=>"us-east-1", :transport_options=>{:request=>{:open_timeout=>0, :timeout=>60}, :proxy=>nil, :headers=>{"Content-Type"=>"application/json"}}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::AWS}, :error_message=>"search-pbc-prod-qlkounnhsmutmhmqzqtjdpvjfu.us-east-1.es.amazonaws.com:443 failed to respond", :error_class=>"Faraday::ClientError", :backtrace=>nil}

[2021-04-01T22:50:07,629][WARN ][logstash.outputs.amazones] Failed to flush outgoing items {:outgoing_count=>1, :exception=>"Faraday::ClientError", :backtrace=>nil}

i got this on logstash log file but this was not repeated

@mtojek And i got
Null exeption error on application server /etc/filebeat-stderror.log

Exception in thread "Thread-2" java.lang.NullPointerException
at org.jruby.util.JRubyClassLoader$1.run(JRubyClassLoader.java:119)
Exception in thread "Thread-2" java.lang.NullPointerException
at org.jruby.util.JRubyClassLoader$1.run(JRubyClassLoader.java:119)

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.