I had enabled authentication on my ELK stack (single node) , every thing works as expected (role,users etc). however my elasticsearch logs have logged some errors such as
[2021-07-06T15:33:59,086][INFO ][o.e.x.s.a.AuthenticationService] [dev-eng05] Authentication of [kibana_system] was terminated by realm [reserved] - failed to authenticate user [kibana_system]
I have confirmed the credentials via curl to be correct
root@dev-eng05:/etc/logstash# curl -u kibana_system 'http://172.26.205.249:9200/_xpack/security/_authenticate?pretty' Enter host password for user 'kibana_system': { "username" : "kibana_system", "roles" : [ "kibana_system" ], "full_name" : null, "email" : null, "metadata" : { "_reserved" : true }, "enabled" : true, "authentication_realm" : { "name" : "reserved", "type" : "reserved" }, "lookup_realm" : { "name" : "reserved", "type" : "reserved" }, "authentication_type" : "realm" } root@dev-eng05:/etc/logstash#
*# If your Elasticsearch is protected with basic authentication, these settings provide*
*# the username and password that the Kibana server uses to perform maintenance on the Kibana*
*# index at startup. Your Kibana users still need to authenticate with Elasticsearch, which*
*# is proxied through the Kibana server.*
***elasticsearch.username*** *:* *"kibana_system"*
***elasticsearch.password*** *:* *"Test123"*
When trying to run my logstash
[2021-07-06T15:42:19,047][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error {:url=>"http://172.26.205.249:9200/", :exception=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :message=>"Got response code '401' contacting Elasticsearch at URL 'http://172.26.205.249:9200/'"}