Unable to authenticate user [kibana_system] for REST request [/_xpack] error

Elastic Stack Kibana
1

I am using EFK stack on Kubernetes, Elastic and Kibana 8.7.0:

  1. I changed the passwords for built-in users elastic and kibana_system with ./bin/elasticsearch-reset-password user --username kibana_system and ./bin/elasticsearch-reset-password user --username elastic
  2. Then I added them to the elasticsearch-keystore with ./bin/elasticsearch-keystore add kibana_pass and ./bin/elasticsearch-keystore add elastic_pass
  3. I have created Secretc with both passwords
    And everything is working fine, but after restart I get this error in my kibana logs:
[2023-05-15T06:54:11.417+00:00][ERROR][plugins.security.authentication] License is not available, authentication is not possible.
[2023-05-15T06:54:11.429+00:00][WARN ][plugins.licensing] License information could not be obtained from Elasticsearch due to ResponseError: security_exception
        Root causes:
                security_exception: unable to authenticate user [kibana_system] for REST request [/_xpack] error

and this is my elastic log:

{"@timestamp":"2023-05-15T09:25:40.548Z", "log.level": "INFO", "message":"Authentication of [kibana_system] was terminated by realm [reserved] - failed to authenticate user [kibana_system]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch-client][transport_worker][T#7]","log.logger":"org.elasticsearch.xpack.security.authc.RealmsAuthenticator","trace.id":"5451638a596cc9a373336ecc3552088a","elasticsearch.cluster.uuid":"6h24ceJZS6-w0WZ3EPg8PA","elasticsearch.node.id":"Cu3XJpMASA20AMH2QcnvoQ","elasticsearch.node.name":"elasticsearch-client","elasticsearch.cluster.name":"elasticsearch"}

And I need to change the passwords again and create new Secrets so I can authenticate both users, I need help how to make them persistent so I dont have to do this everytime a node restarts

2

Elasticsearch keystore is not the place for storing these passwords. What are you trying to achieve? Maybe you were looking for kibana-keystore?

3

Hi, I want to make the passwords persistent, so when the cluster fails or restarts they still to remain the same.

4

It's persistant already since ./bin/elasticsearch-reset-password saves the new password to the security index which is on disk and available across restart (assuming you don't actively delete data between restarts).

1 Like
5

I don't delete data, but when my cluster fails or I restart it, I get the logs above and I need to reset the password again so elastic can authenticate the user kibana_system

6

Did you found any solution for that problem?
I have the same problem and kibana doesn´t start. It shows your error, too.

7

No, I still haven't found a solution for this problem.
If I solve it I will write here

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.