HI,
My input is kafka and output is a file.
Sometimes i have seen the logstash process just hangs(doing nothing).
To debug this issue i enabled verbose logging and started the process.
After few days of running, it hanged again with below error in the log:
{:timestamp=>"2015-10-23T00:49:18.263000-0600", :message=>"Exception in filterworker", "exception"=>#<RuntimeError: can't add a new key into hash during iteration>, "backtrace"=>["org/jruby/RubyHash.java:991:in []='", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/util/accessors.rb:119:infind_or_create_target'", "org/jruby/RubyArray.java:1613:in each'", "org/jruby/RubyEnumerable.java:866:ininject'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/util/accessors.rb:119:in find_or_create_target'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/util/accessors.rb:98:inlookup_or_create'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/util/accessors.rb:63:in set'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/event.rb:146:in[]='", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-urldecode-1.0.0/lib/logstash/filters/urldecode.rb:36:in filter'", "org/jruby/RubyHash.java:1341:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-urldecode-1.0.0/lib/logstash/filters/urldecode.rb:36:in filter'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/filters/base.rb:163:inmulti_filter'", "org/jruby/RubyArray.java:1613:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/filters/base.rb:160:inmulti_filter'", "(eval):155:in filter_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:219:infilterworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:157:in `start_filters'"], :level=>:error}
Can someone please help understanding whats going on here?
Thanks