I have enabled inter-node TLS communication and xpack security as described here:
I have set passwords for elastic, kibana and all other users with
Blockquote
elasticsearch-setup-passwords interactive
Blockquote
i have checked with user elastic several API endpoints (_cluster, nodes, shards etc..) and i got proper response.
i run kibana app in docker on each elasticsearch node ( master and datanode) and I have load balancer to distribute requests between 3 these nodes. I have enabled xpack security in kibana.yml either.
Right now when i try o access URL of load balancer I see normal kibana login prompt, but it does not let me in. I tried to login with user elastic and kibana. I don`t get any error message, just login prompt again. Have to notice: if i use intentionally incorrect password for user 'elastic' i receive error message about 'incorrect user/password', but when i use the correct one I just observe kibana logo moving around itself several seconds like it doing something, but then i return to the login prompt again. No error in kibana log.
I see in elasticearch log the following errors(?):
Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
elasticsearch and kibana versions: 7.1.1
kibana config:
Blockquote
server.name: es-master-3
server.host: 0.0.0.0
xpack.security.enabled: true
elasticsearch.hosts: "http://127.0.0.1:9200"
elasticsearch.username: "kibana"
elasticsearch.password: "kibana"
xpack.license_management.enabled: True
xpack.monitoring.ui.container.elasticsearch.enabled: True
Blockquote
Please give me some directions.
Update: I have enabled verbose logging on kibana and this is what i get:
Blockquote
log [19:29:50.720] [debug][plugin] Checking Elasticsearch version
log [19:29:51.713] [debug][legacy-proxy] Event is being forwarded: connection
log [19:29:51.713] [debug][legacy-service] Request will be handled by proxy GET:/app/kibana.
error [19:29:51.714] [debug][auth][security][session] Error: Unauthorized
at validate (/usr/share/kibana/node_modules/hapi-auth-cookie/lib/index.js:153:49)
at Object.authenticate (/usr/share/kibana/node_modules/hapi-auth-cookie/lib/index.js:226:26)
at module.exports.internals.Manager.execute (/usr/share/kibana/node_modules/hapi/lib/toolkit.js:35:106)
at module.exports.internals.Auth.test (/usr/share/kibana/node_modules/hapi/lib/auth.js:92:54)
at Session.get (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/authentication/session.js:56:47)
at Authenticator.authenticate (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/authentication/authenticator.js:132:49)
at Object.server.expose.request [as authenticate] (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/authentication/authenticator.js:288:60)
at Object.authenticate (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/auth_redirect.js:28:60)
at module.exports.internals.Manager.execute (/usr/share/kibana/node_modules/hapi/lib/toolkit.js:35:106)
at module.exports.internals.Auth._authenticate (/usr/share/kibana/node_modules/hapi/lib/auth.js:238:58)
at authenticate (/usr/share/kibana/node_modules/hapi/lib/auth.js:214:21)
at Request._lifecycle (/usr/share/kibana/node_modules/hapi/lib/request.js:263:62)
at process._tickCallback (internal/process/next_tick.js:68:7)
I assume this is somehow related to fact that kibana runs in docker container, and elasticsearch runs as application on the server (installed from deb package)
User/password for kibana user seems to be correct
root@es-master-1:/srv/kibana# curl -u kibana 'http://localhost:9200/_xpack/security/_authenticate?pretty'
Enter host password for user 'kibana':
{
"username" : "kibana",
"roles" : [
"kibana_system"
],
"full_name" : null,
"email" : null,
"metadata" : {
"_reserved" : true
},
"enabled" : true,
"authentication_realm" : {
"name" : "reserved",
"type" : "reserved"
},
"lookup_realm" : {
"name" : "reserved",
"type" : "reserved"
}
}