Elasticsearch input plugin (from logstash 5.3.2) doesn't work with ES 1.7

Elastic Stack Logstash
1

According to the release notes of LS 5.3.2 https://www.elastic.co/guide/en/logstash/current/logstash-5-3-2.html#_input_plugins_2
the input plugin is now trying to use scroll in json body of the request, however as I understand Elasticsearch 1.7 only supports scroll as a URL parameter.

As a result I get these errors on LS 5.3.2:

[2017-05-03T19:46:37,137][ERROR][l.pipeline               ] A plugin had an unrecoverable error. Will restart this plugin.
  Plugin: <LogStash::Inputs::Elasticsearch hosts=>["elasticsearch:9280"], index=>"logs-2017.03.01", size=>500, docinfo=>true, id=>"3df92c2f0e895e5fe8078702bd79b400b0cb9e97-1", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_05b9ffec-cd1b-4208-beeb-e2d2d3b491dd", enable_metric=>true, charset=>"UTF-8">, query=>"{ \"sort\": [ \"_doc\" ] }", scroll=>"1m", docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"], ssl=>false>
  Error: [400] {"error":"ElasticsearchIllegalArgumentException[Failed to decode scrollId]; nested: IOException[Bad Base64 input character decimal 123 in array position 0]; ","status":400}
  Exception: Elasticsearch::Transport::Transport::Errors::BadRequest
  Stack: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:202:in `__raise_transport_error'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/base.rb:319:in `perform_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-5.0.4/lib/elasticsearch/transport/client.rb:131:in `perform_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-5.0.4/lib/elasticsearch/api/actions/scroll.rb:61:in `scroll'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.3/lib/logstash/inputs/elasticsearch.rb:197:in `scroll_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.3/lib/logstash/inputs/elasticsearch.rb:166:in `process_next_scroll'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-elasticsearch-4.0.3/lib/logstash/inputs/elasticsearch.rb:158:in `run'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:425:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:419:in `start_input'

If I use Logstash 5.2.2 - everything's working properly.

Am I doing something wrong or is there a workaround?
Does logstash 5.x officially support Elasticsearch 1.x as input?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.