Hi, I am trying to automate Elasticsearch 7.16 cluster deployment, including system user passwords. For that I'd add bootstrap.password to elastic keystore and use that password to set the permanent password for elastic user. But the password I assign to the keystore does not seem to work, as change password API returns 401 "unable to authenticate user [elastic] for REST request". Why is it unable to authenticate?
Here's my Elasticsearch.yml file:
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
node.roles: [ master ]
cluster.name: cluster_elastic
node.name: ${HOSTNAME}
http.port: 9200
transport.port: 9300
network.host: _eth0:ipv4_
network.bind_host: [ "_eth0:ipv4_", "localhost4" ]
discovery.seed_hosts: [ "master1_ip", "master2_ip", "master3_ip" ]
cluster.initial_master_nodes: [ "master-1", "master-2", "master-3" ]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
The sequence of automation is the following:
First master node is created with the config above, TSL key/truststore passwords are inserted into the keystore and Elasticsearch service is started. Next the second master node is created with the config above, bootstrap.password is inserted into the keystore followed by TSL key/truststore passwords, Elasticsearch service is started which enables the cluster, then the API call is made to change elastic password.
I attempted to do this manually also, by stopping the Elasticsearch service on the first master node, adding the bootstrap.password, starting the service and making the API call:
[root@master-1 ~]# systemctl stop elasticsearch
[root@master-1 ~]# echo "elastic" | /usr/share/elasticsearch/bin/elasticsearch-keystore add -x "bootstrap.password"
[root@master-1 ~]# /usr/share/elasticsearch/bin/elasticsearch-keystore list
bootstrap.password
keystore.seed
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password
[root@master-1 ~]# systemctl start elasticsearch
[root@master-1 ~]# curl -u "elastic:elastic" -X GET http://127.0.0.1:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@master-1 ~]#
[root@master-1 ~]# curl -u "elastic:elastic" -XPOST "http://127.0.0.1:9200/_xpack/security/user/elastic/_password" -d '{"password":"NEW_PASSWORD"}' -H "Content-Type:application/json"
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_security/user/elastic/_password]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_security/user/elastic/_password]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
What am I doing wrong?