Hi i'm handling request rate degradation and I noticed that some of my nodes behaving strange , exactly at 03:04 every night request rate Either drops from 900 to 200 or climb up from 200 to 900
I'm not sure this is normal behavior
Hi i'm handling request rate degradation and I noticed that some of my nodes behaving strange , exactly at 03:04 every night request rate Either drops from 900 to 200 or climb up from 200 to 900
I'm not sure this is normal behavior
Hi guys ,any Ideas ?
We aren't all guys
What do your Elasticsearch logs for the nodes in question show at this time?
, you are right , sorry
I found the problem , logstash server is bounces with
multiline exception
[2020-08-11T00:58:25,215][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method to_hash_with_metadata' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-multiline-3.0.4/lib/logstash/filters/multiline.rb:273:in
block in merge'", "org/jruby/RubyArray.java:1734:in each'", "org/jruby/RubyEnumerable.java:936:in
inject'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-multiline-3.0.4/lib/logstash/filters/multiline.rb:272:in merge'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-multiline-3.0.4/lib/logstash/filters/multiline.rb:202:in
block in flush'", "org/jruby/RubyHash.java:1343:in each'", "org/jruby/RubyEnumerable.java:830:in
map'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-multiline-3.0.4/lib/logstash/filters/multiline.rb:200:in flush'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:60:in
block in define_flush_method'", "(eval)in block in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:602:in
block in flush_filters'", "org/jruby/RubyArray.java:1734:in each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:601:in
flush_filters'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:642:in flush_filters_to_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:424:in
worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:384:in `block in start_workers'"]}
the condition in filter looks like :
if [logger] == "stderr" {
multiline {
pattern => "^\t"
what => previous
}
}
please advice what can be done to avoid logstash bounce .
It might be better to post something in the Logstash category with a bit more detail.
I will , Thank you
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.