Logstash 5 Alpha4 to elasticsearch5 Alpha4 communication error

Elastic Stack Logstash
1

elasticsearch 5 is secured with xpack security and hooked with ldap which is working fine. Even user has admin right in role_mapping.

Below is configuration
output {
elasticsearch {
hosts => ['localhost:9200']
user => 'gaurav@gmail.com'
password => 'pwd'
}
}

Getting below error and because of which logstash is not able to pass data to elasticsearch.

{:timestamp=>"2016-07-14T16:32:29.592000+0530", :message=>"Encountered an unexpected error submitting a bulk request! Will retry.", :error_message=>"undefined method code' for #LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError:0x97f8a30", :class=>"NoMethodError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:217:insafe_bulk'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:105:in submit'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:72:inretrying_submit'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:23:in multi_receive'", "org/jruby/RubyArray.java:1653:ineach_slice'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb:22:in multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:136:inthreadsafe_multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_

2

Hi Gaurav,

A typo in the Logstash elasticsearch output is preventing you from seeing what the real problem is in this case.
Are you able to edit one of the ruby files in your install?

If so, you will need to open this file...

/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.1-java/lib/logstash/outputs/elasticsearch/common.rb

go to line 217.
at the very end of the line, change e.code to e.response_code

BTW it looks like Elasticsearch is returning a response code that we have not anticipated.

Look out for an error line in the Logstash logs that contains...
Got a bad response code from server, but this code is not considered retryable

Post the full line here.

Good luck.

3

Alternatively, this typo was fixed in v 4.1.3
Do...
bin/logstash-plugin update logstash-output-elasticsearch

4

{:timestamp=>"2016-08-18T02:38:07.483000+0000", :message=>"Encountered an unexpected error submitting a bulk request! Will retry.", :error_message=>"Cabin::Channel#error(message, data={}) - second argument you gave me wasa Array, I require a Hash.", :class=>"ArgumentError", :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/cabin-0.8.1/lib/cabin/mixins/logger.rb:72:in error'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:75:in retrying_submit'", "org/jruby/RubyArray.java:1613:in each'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:75:in retrying_submit'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:23:in multi_receive'", "org/jruby/RubyArray.java:1653:in each_slice'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-4.1.3-java/lib/logstash/outputs/elasticsearch/common.rb:22:in multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:136:in threadsafe_multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:122:in multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:291:in output_batch'", "org/jruby/RubyHash.java:1342:in each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:291:in output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:in worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:215:in start_workers'"], :level=>:error}

How to resolve this error?
Thanks!

5

Having here same issue, did you found a solution ?