Authenticating with built-in users

[zirota (Hazzen Chua)]

Hello, I ran into some problems with authenticating with built-in users (in this case elastic) on my cluster nodes other than the master.

I have a cluster setup on 8.5.3:

• 3 master nodes
• 4 data nodes
• 2 coordinator nodes

I have done:

1. First time interactive password setup for built in users
2. Added xpack.security.authc.realms.native.native1.order: 0 in config

Other than the master, none of the other nodes are able to authenticate the elastic user.
The log shows that:

Authentication of [elastic] was terminated by realm [reserved] was terminated by realm [reserved] - failed to authenticate user [elastic]

This is tested via simple http requests to https://<node-hostname>:9200 with the elastic user credentials.

Is this behavior intended?

---

I have also tested that the nodes are able to authenticate these other types of users with no issues:

1. Native users - created via `/_security/name/
2. AD users (after adding license)

[Yang_Wang]

No it's not the intended behaviour especially if you have done "First time interactive password setup for built in users".

What version are you using? If it is 8.0+, you can reset elastic's password with the elasticsearch-reset-password CLI.

[zirota (Hazzen Chua)]

Very thankful for the suggestion, which helped to resolve the issue. I am running v8.5.3.

In my case, I created a new superuser account via api.
Then sign in to Kibana using said user and performed a password reset for all built-in users.
Tested and verified that built-in users is able to authenticate correctly from other nodes.

Any idea what might be the root cause?