Blocked outputs

Elastic Stack Logstash
1

Recently I had an issue where my Elastic Cloud instance was not responding well. Logstash was throwing a lot of errors about not being able to send events, and I noticed that some of my other outputs weren't getting processed, specifically critical exec outputs.

I'm running logstash-5.0.0-alpha5 which was upgraded from 2.3 and the elasticsearch-output version 4.1.3 which I think is 8 or 9 months old.

If one output is blocked is it expected that others will be? That the whole pipeline will be blocked?
I wondering if this is related to outdated and alpha versions of the products or if this behavior is expected, and I should design my systems to handle this.

Here's the log:

{
  :timestamp=>"2017-03-12T22:44:35.915000-0700",
  :message=>"Encountered an unexpected error submitting a bulk request! Will retry.",
  :error_message=>"Cabin::Channel#error(message, data={}) - second argument you gave me wasa Array, I require a Hash.",
  :class=>"ArgumentError",
  :backtrace=>[
    "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/cabin-0.8.1/lib/cabin/mixins/logger.rb:72:in `error'",
    "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:75:in `retrying_submit'",
    "org/jruby/RubyArray.java:1613:in `each'",
    "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:75:in `retrying_submit'",
    "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:23:in `multi_receive'",
    "org/jruby/RubyArray.java:1653:in `each_slice'",
    "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:22:in `multi_receive'",
    "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:136:in `threadsafe_multi_receive'",
    "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:122:in `multi_receive'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:291:in `output_batch'",
    "org/jruby/RubyHash.java:1342:in `each'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:291:in `output_batch'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in `worker_loop'",
    "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:215:in `start_workers'"
  ],
  :level=>:error
}
2

If one output is blocked is it expected that others will be? That the whole pipeline will be blocked?

Yes, this is expected.

3

Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.