Elasticsearch error when trying to run bin/elasticsearch-setup-passwords

Elastic Stack Elasticsearch
1

I am setting up Elasticsearch version 7.17.11 on Ubuntu 20.04. It will be a single instance of Elasticsearch but I will have separate instances for Kibana and Logstash. I am trying to recreate our setup in another cloud project without using a cluster for Elasticsearch and those settings are the only differences from a working build as far as I am aware.

This is the error when I run the bin/elasticsearch-setup-passwords tool:

Failed to authenticate user 'elastic' against https://localhost:9200/_security/_authenticate?pretty
Possible causes include:
 * The password for the 'elastic' user has already been changed on this cluster
 * Your elasticsearch node is running against a different keystore
   This tool used the keystore at /etc/elasticsearch/elasticsearch.keystore


ERROR: Failed to verify bootstrap password

The keystore does appear to be at /etc/elasticsearch/elasticsearch.keystore. The password cannot have been already changed because I create a new instance with Terraform every time and run an Ansible playbook against it.

This is my elasticsearch.yml file:

node.name: test-elasticsearch
network.host: localhost
http.port: 9200
discovery.type: single-node

# Transport layer
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.transport.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt

# HTTP layer
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.http.ssl.certificate_authorities: /etc/elasticsearch/certs/ca/ca.crt

# Elasticsearch authentication
xpack.security.enabled: true
#xpack.security.enabled: false

path.data: /mnt/elastic_data_disk
path.logs: /mnt/elastic_data_disk

The certs have been setup with the bin/elasticsearch-certutil tool and put in the correct paths BTW.

This is the instances.yml file I used to configure the certs BTW (with full public IP - censored here):

instances:
- name: "test-elasticsearch"
  ip:
  - "35.xx.xx.xx"

Any help is appreciated: I have read all of the related Elastic documentation I am aware of that relates and I am stuck and cannot find anything online.

2

Hi @Kris_U

How did you install?

Curious Do you have a keystore somewhere in /usr/share/elasticsearch

3

Not entirely sure but the error message you're seeing suggests that the 'elastic' user cannot authenticate against the Elasticsearch instance. This could be due to a few reasons, but let's start with the most common ones:

  1. The password for the 'elastic' user has already been changed: You mentioned that you create a new instance every time, so this is unlikely to be the issue. However, it's worth double-checking to make sure that no scripts or processes are changing the password before you run the elasticsearch-setup-passwords tool.

  2. The Elasticsearch node is running against a different keystore: This could be the issue if your Elasticsearch instance is not using the keystore at /etc/elasticsearch/elasticsearch.keystore. You can verify this by checking the elasticsearch.yml configuration file and making sure that the xpack.security.http.ssl.key and xpack.security.http.ssl.certificate paths are correct.

  3. Incorrect SSL configuration: Your elasticsearch.yml file shows that you have SSL enabled for both the transport and HTTP layers. If the SSL certificates or keys are incorrect or not properly configured, this could prevent the 'elastic' user from authenticating. You can verify the SSL configuration by checking the certificate paths in the elasticsearch.yml file and making sure that the certificates and keys are correctly generated and placed in the specified paths.

  4. Network or firewall issues: If your Elasticsearch instance is not accessible at localhost:9200 due to network or firewall issues, this could also prevent the 'elastic' user from authenticating. You can test this by trying to access Elasticsearch at localhost:9200 from the machine where you're running the elasticsearch-setup-passwords tool.

If none of the above suggestions resolve the issue, you might want to consider enabling debug logging for the security module to get more detailed information about the authentication failure. You can do this by adding the following line to your log4j2.properties file:

logger.security.name = org.elasticsearch.xpack.security
logger.security.level = debug

After making this change, restart Elasticsearch and try running the elasticsearch-setup-passwords tool again. The debug logs should provide more information about why the 'elastic' user cannot authenticate.

disclaimer I also used opsgpt.io to write part of this answer

4

Thank you! I will try the suggested. Where will the debug logs be?

5

Good suggestion! I just checked and no .keystore files in /usr/share/elasticsearch (or anywhere else but /etc/elasticsearch for that matter).

6

I did the Debian repo package install from: https://artifacts.elastic.co/packages/7.x/apt

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.