ES 7.14: can not get API key to work

Elastic Stack Elasticsearch
1

I am trying to use an API key to authenticate using the API:

first the code then the stack trace...

      $logger.info "Try connecting to ES #{params}" if $logger
      begin
        @client =  Elasticsearch::Client.new params


 INFO  syslog-es-connect-loghost : Try connecting to ES {:hosts=>[{"host"=>"secmonprd08.its.auckland.ac.nz", "scheme"=>"https", "port"=>9200}], :request_timeout=>10, :randomize_hosts=>true, :api_key=>"..."}
Traceback (most recent call last):
	19: from /usr/local/tools/dev/siem_logging/bin/syslog-ng-es.rb:193:in `<main>'
	18: from /usr/local/tools/dev/siem_logging/bin/syslog-ng-es.rb:193:in `new'
	17: from /usr/local/tools/dev/siem_logging/lib/source/loghost.rb:36:in `initialize'
	16: from /usr/local/tools/dev/siem_logging/lib/source.rb:49:in `initialize'
	15: from /usr/local/tools/dev/siem_logging/lib/source.rb:92:in `open_es'
	14: from /usr/local/tools/dev/siem_logging/lib/source.rb:92:in `new'
	13: from /usr/local/tools/dev/common-library/lib/es_connection.rb:51:in `initialize'
	12: from /usr/local/tools/dev/common-library/lib/es_connection.rb:103:in `connect'
	11: from /usr/local/tools/dev/common-library/lib/es_connection.rb:103:in `new'
	10: from /var/lib/gems/2.7.0/gems/elasticsearch-7.15.0/lib/elasticsearch.rb:35:in `initialize'
	 9: from /var/lib/gems/2.7.0/gems/elasticsearch-7.15.0/lib/elasticsearch.rb:35:in `new'
	 8: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/client.rb:171:in `initialize'
	 7: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/client.rb:171:in `new'
	 6: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:60:in `initialize'
	 5: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:149:in `__build_connections'
	 4: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:149:in `map'
	 3: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:157:in `block in __build_connections'
	 2: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/http/faraday.rb:63:in `__build_connection'
	 1: from /var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:237:in `__full_url'
/var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:237:in `+': no implicit conversion of nil into String (TypeError)

lots of fiddling with the debugger established that Faraday was expecting a URL parameter and died when it was nil.

I then reworked the code to use these parameters:

{:hosts=>[{:host=>"https://secmonprd08.its.auckland.ac.nz:9200"}], :request_timeout=>10, :randomize_hosts=>true, :api_key=>"..."}

this time I got a client but when I tried @health = @client.cluster.health I got another crash.

More work in the debugger yeilded:

/var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:286:            response = block.call(connection, url)
(rdb:1) p url
"http://https://secmonprd08.its.auckland.ac.nz:9200:9200/"

which, unsurprisingly, causes a crash later on. However if I use the debugger:

(rdb:1) url =  "https://secmonprd08.its.auckland.ac.nz:9200"
"https://secmonprd08.its.auckland.ac.nz:9200"
...........
/var/lib/gems/2.7.0/gems/elasticsearch-transport-7.15.0/lib/elasticsearch/transport/transport/base.rb:344:          took     = (json['took'] ? sprintf('%.3fs', json['took']/1000.0) : 'n/a') rescue 'n/a'
(rdb:1) p json
{"name"=>"secmonprd08", "cluster_name"=>"test", "cluster_uuid"=>"mSgZQOoHRn-ttSYsjo60GA", "version"=>{"number"=>"7.14.0", "build_flavor"=>"default", "build_type"=>"deb", "build_hash"=>"dd5a0a2acaa2045ff9624f3729fc8a6f40835aa1", "build_date"=>"2021-07-29T20:49:32.864135063Z", "build_snapshot"=>false, "lucene_version"=>"8.9.0", "minimum_wire_compatibility_version"=>"6.8.0", "minimum_index_compatibility_version"=>"6.0.0-beta1"}, "tagline"=>"You Know, for Search"}

it works.

Any clues as to what is going on?

Sharp eyes may notice that I said that the ES version is 7.15 but the API version is 7.15. I noticed this while I was writing this up and went back and installed the 7.14 version of the gem and verified it performed the same.

I have had this code working fine for some time without https and the API key using the hash form of the parameters

2

Amazing what a nights sleep does to one's perceptions!

This morning I did what I should have done at the start and abstracted the problem down to a single line program:

require 'elasticsearch'

@client =  Elasticsearch::Client.new {:hosts=>[{"host"=>"secmonprd08.its.auckland.ac.nz", "scheme"=>"https", "port"=>9200}], :request_timeout=>10, :randomize_hosts=>true, :api_key=>"..."}

running this gives exactly the same crash as the before. Then I noticed that the keys if the hosts hash were stings not symbols! They were inserted into to the arguments hash from another hash which originates as a json file.

client =  Elasticsearch::Client.new hosts:[{host: "secmonprd08.its.auckland.ac.nz", scheme: "https", port: 9200}], :request_timeout=>10, :randomize_hosts=>true, :api_key=>"..."

works as expected.

I need to convert the keys of that hash to symbols before using it in parameters.