Elasticsearch-reset-password returns error code 69

I have installed elasticsearch-8.15.3-1.x86_64 by following Install Elasticsearch with RPM | Elasticsearch Guide [8.17] | Elastic

and I verified it is installed properly by

$ sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u 
elastic:$ELASTIC_PASSWORD https://localhost:9200 
{
  "name" : "ip-10-0-92-200.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "s6A7rgdyTxqPI6PBbcfN3g",
  "version" : {
    "number" : "8.15.3",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "f97532e680b555c3a05e73a74c28afb666923018",
    "build_date" : "2024-10-09T22:08:00.328917561Z",
    "build_snapshot" : false,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

However, when I tried to reset the password for the user elastic, I got this error

$ sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

ERROR: Failed to determine the health of the cluster. , with exit code 69

so it seems the script somehow couldn't determine the health of the cluster, but if I query the health endpoint, the cluster seems to be healthy

sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200/_cluster/health
{"cluster_name":"elasticsearch","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":1,"active_shards":1,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}

--url
Specifies the base URL (hostname and port of the local node) that the tool uses to submit API requests to Elasticsearch. The default value is determined from the settings in your elasticsearch.yml file. If xpack.security.http.ssl.enabled is set to true, you must specify an HTTPS URL.

Try....

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic --url https://localhost:9200

Thanks for the quick response Stephen, got the same error though

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic --url https://localhost:9200

ERROR: Failed to determine the health of the cluster. , with exit code 69

can you try to cd to the directory first...

cd /usr/share/elasticsearch
./bin/elasticsearch-reset-password -u elastic --url https://localhost:9200

tried it, but got the same error

Try ---verbose it should show you want the actual error is.

1 Like

Hi Tim, here is the output with the verbose flag, seems to be the same

sudo ./bin/elasticsearch-reset-password -u elastic --url https://localhost:9200 --verbose

ERROR: Failed to determine the health of the cluster. , with exit code 69

So the last time I ran into this the elastic keystore was in the wrong place or the certificates had been changed post install and the the values in the elastic keystore were wrong

can you share you elasticsearch.yml please?

can you list the keys in the elastic keystore?

Did you alter any of the certificates?

can you run the curl with the -v option please and show the results

Hi Stephen,

I didn't change anything in the elasticsearch.yml or the elastic keystore

here is the content of the yml file

$ sudo cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically      
# generated to configure Elasticsearch security features on 21-01-2025 23:28:29
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ip-10-0-64-175.us-east-2.compute.internal"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

and here the curl -v output

$ sudo curl -v --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWOR

*   Trying ::1:9200...
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/elasticsearch/certs/http_ca.crt
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Unknown (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=ip-10-0-64-175.us-east-2.compute.internal
*  start date: Jan 21 23:28:34 2025 GMT
*  expire date: Jan 21 23:28:34 2027 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elasticsearch security auto-configuration HTTP CA
*  SSL certificate verify ok.
* Server auth using Basic with user 'elastic'
* TLSv1.2 (OUT), TLS header, Unknown (23):
> GET / HTTP/1.1
> Host: localhost:9200
> Authorization: Basic ZWxhc3RpYzoyQlJwWXJYa01lME92YWNtZGpyLQ==
> User-Agent: curl/7.76.1
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Unknown (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Unknown (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 566
< 
{
  "name" : "ip-10-0-64-175.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Zgt7KE3-Rt2twxvL65pQYg",
  "version" : {
    "number" : "8.13.4",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "da95df118650b55a500dcc181889ac35c6d8da7c",
    "build_date" : "2024-05-06T22:04:45.107454559Z",
    "build_snapshot" : false,
    "lucene_version" : "9.10.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host localhost left intact

could you tell me how to list keys in the elastic keystore?

ok then try this...

sudo ./bin/elasticsearch-reset-password -u elastic --url https://ip-10-0-64-175.us-east-2.compute.internal:9200 --verbose

Not sure why you chose to install 8.15.3 following the 8.17 instructions, but thats just a curiosity .... not important here, though it actually says 8.13.4 below ?

Thinking a bit out of the elasticsearch-reset-password box ...

Your current elastic password is working, right, cos we see the

{
  "name" : "ip-10-0-64-175.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Zgt7KE3-Rt2twxvL65pQYg",
  "version" : {
    "number" : "8.13.4",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "da95df118650b55a500dcc181889ac35c6d8da7c",
    "build_date" : "2024-05-06T22:04:45.107454559Z",
    "build_snapshot" : false,
    "lucene_version" : "9.10.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

though above it was:

{
  "name" : "ip-10-0-92-200.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "s6A7rgdyTxqPI6PBbcfN3g",
  "version" : {
    "number" : "8.15.3",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "f97532e680b555c3a05e73a74c28afb666923018",
    "build_date" : "2024-10-09T22:08:00.328917561Z",
    "build_snapshot" : false,
    "lucene_version" : "9.11.1",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

output from curl.

But you just want to change that password? Do you have kibana installed and working. If so, can you not change the password there?

You sure you are using the same instances?

"name" : "ip-10-0-92-200.us-east-2.compute.internal",
or
"name" : "ip-10-0-64-175.us-east-2.compute.internal",

They also have different cluster_uuid ?

I think there is something going on here you might have either overlooked, or failed to mention.

Hi Kevin,

Not sure why you chose to install 8.15.3 following the 8.17 instructions
The instructions for both 8.15.3 and 8.17 seem to be the same except for the pkg version. I have to install 8.15.3 for some compatibility reason

You sure you are using the same instances?
No, I teared down the old instance and created a new one for a fresh start. However, I did install elasticsearch the same way, so I don't think that matters, but I should have mentioned that to avoid confusion.

sudo ./bin/elasticsearch-reset-password -u elastic --url https://ip-10-0-84-175.us-east-2.compute.internal:9200 --verbose

ERROR: Failed to determine the health of the cluster. , with exit code 69

Note that I am testing with a new instance, so the ip is different

Hi folks, I sort of find out the issue, it seems there is something wrong with the 9200 port on our instance. If I change the port to 9000, it works. Thanks for your help. Feel free to close this question.

I am glad you sorted out the issue, port 9000 vs 9200. Really? Weird, but whatever.

Also

I have to install 8.15.3 for some compatibility reason

OK, slightly dubious given you had installed 8.13.5 at least once, but ...

I did install elasticsearch the same way, so I don't think that matters,

If it was really "the same" you would not have ended up with different versions installed on different instances - the 2 versions you shared the output:

{
  "name" : "ip-10-0-64-175.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Zgt7KE3-Rt2twxvL65pQYg",
  "version" : {
    "number" : "8.13.4",

and

{
  "name" : "ip-10-0-92-200.us-east-2.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "s6A7rgdyTxqPI6PBbcfN3g",
  "version" : {
    "number" : "8.15.3",

but I should have mentioned that to avoid confusion.

A really hard aspect, from my side trying to help people here, is so often key information is not shared, because "it doesn't matter". But we look at the stuff that was shared, and it doesn't tally. So we have to try to figure out the gaps in the story being told, which are often NOT important, but DO confuse.

1 Like