Unable to authenticate user [elastic] for REST request [/]

I'm trying to deploy Elastic Stack as described here: https://www.elastic.co/blog/getting-started-with-the-elastic-stack-and-docker-compose.
Docker images start correctly, but I can't authenticate although the password of the elastic user on the curl-based command line matches the one set in the .env file:

$ curl -vvvv --cacert ../ca.crt -u elastic:0123456789 https://localhost:9200
* Host localhost:9200 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9200...
* Connected to localhost (::1) port 9200
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: ../ca.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / RSASSA-PSS
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=es01
*  start date: Dec 23 19:10:50 2024 GMT
*  expire date: Dec 23 19:10:50 2027 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: localhost:9200
> Authorization: Basic ZWxhc3RpYzowMTIzNDU2Nzg5
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 401 Unauthorized
* Authentication problem. Ignoring this.
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< content-type: application/json
< content-length: 465
< 
* Connection #0 to host localhost left intact
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}

This is the log of the container:

es01-1          | {"@timestamp":"2024-12-24T19:29:20.269Z", "log.level": "INFO", "message":"Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][system_critical_read][T#1]","log.logger":"org.elasticsearch.xpack.security.authc.RealmsAuthenticator","elasticsearch.cluster.uuid":"VLToNhn1R5idrlmJPqH01A","elasticsearch.node.id":"S4wK0tETSmmp-nYQZZa3MA","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"docker-cluster"}

Any help would be really appreciated.

Is that what you set in the .env file when you first ran the compose?

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=changeme

The very first time I used a different password, but I encountered OOM-related errors causing the elasticsearch container to be killed. Then,

  • I freed some memory
  • I manually removed all the containers
  • I changed the password (see .env file below)
  • I rebuilt the docker images and started them again with docker compose up.

This is the complete .env file:

# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=0123456789

# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=kb0000

# Version of Elastic products
STACK_VERSION=8.7.1

# Set the cluster name
CLUSTER_NAME=docker-cluster

# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic
#LICENSE=trial

# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200

# Port to expose Kibana to the host
KIBANA_PORT=5601

# Increase or decrease based on the available host memory (in bytes)
# 1 GB
ES_MEM_LIMIT=1073741824
KB_MEM_LIMIT=1073741824
LS_MEM_LIMIT=1073741824
# 512 MB
#ES_MEM_LIMIT=536870912
#KB_MEM_LIMIT=536870912
#LS_MEM_LIMIT=536870912


# SAMPLE Predefined Key only to be used in POC environments
ENCRYPTION_KEY=c34d38b3a14956121ff2170e5030b471551370178f43e5626eec58b04a30fae2

It's probably the first password you used the first time because the data volumes were created there, and that's where the password is stored.

So use the first password... or delete all the volumes etc. and start over.

I was so dumb that I forgot to clean up the volumes too! :man_facepalming:

Anyway, I deleted everything and started over, but I still get the same authentication error:

$ docker compose down -v
[+] Running 13/13
 ✔ Container elastic-stack-docker-part-one-filebeat01-1    Removed                                                                                       0.1s 
 ✔ Container elastic-stack-docker-part-one-logstash01-1    Removed                                                                                       0.0s 
 ✔ Container elastic-stack-docker-part-one-metricbeat01-1  Removed                                                                                       0.3s 
 ✔ Container elastic-stack-docker-part-one-kibana-1        Removed                                                                                       0.3s 
 ✔ Container elastic-stack-docker-part-one-es01-1          Removed                                                                                       3.0s 
 ✔ Container elastic-stack-docker-part-one-setup-1         Removed                                                                                       0.1s 
 ✔ Volume elastic-stack-docker-part-one_metricbeatdata01   Removed                                                                                       0.0s 
 ✔ Volume elastic-stack-docker-part-one_certs              Removed                                                                                       0.1s 
 ✔ Volume elastic-stack-docker-part-one_esdata01           Removed                                                                                       0.1s 
 ✔ Volume elastic-stack-docker-part-one_filebeatdata01     Removed                                                                                       0.1s 
 ✔ Volume elastic-stack-docker-part-one_kibanadata         Removed                                                                                       0.0s 
 ✔ Volume elastic-stack-docker-part-one_logstashdata01     Removed                                                                                       0.1s 
 ✔ Network elastic                                         Removed
$ docker compose up
[+] Running 7/5
 ✔ Network elastic                                         Created                                                                    0.1s 
 ✔ Container elastic-stack-docker-part-one-setup-1         Created                                                                    0.0s 
 ✔ Container elastic-stack-docker-part-one-es01-1          Created                                                                    0.0s 
 ✔ Container elastic-stack-docker-part-one-filebeat01-1    Created                                                                    0.0s 
 ✔ Container elastic-stack-docker-part-one-kibana-1        Created                                                                    0.0s 
 ✔ Container elastic-stack-docker-part-one-metricbeat01-1  Created                                                                    0.0s 
 ✔ Container elastic-stack-docker-part-one-logstash01-1    Created                                                                    0.0s 
Attaching to es01-1, filebeat01-1, kibana-1, logstash01-1, metricbeat01-1, setup-1
...
es01-1          | {"@timestamp":"2024-12-26T09:51:50.516Z", "log.level": "INFO", "message":"Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][system_critical_read][T#3]","log.logger":"org.elasticsearch.xpack.security.authc.RealmsAuthenticator","elasticsearch.cluster.uuid":"LZJP7_1HRrC6riNtILwHtg","elasticsearch.node.id":"hFMz2vt6R4GmjuERD5ygvw","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"docker-cluster"}



$ curl -vvvv --cacert ../ca.crt -u 'elastic:es0000' https://localhost:9200
* Host localhost:9200 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9200...
* Connected to localhost (::1) port 9200
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: ../ca.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / RSASSA-PSS
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=es01
*  start date: Dec 26 09:47:54 2024 GMT
*  expire date: Dec 26 09:47:54 2027 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: localhost:9200
> Authorization: Basic ZWxhc3RpYzplczAwMDA=
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 401 Unauthorized
* Authentication problem. Ignoring this.
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< content-type: application/json
< content-length: 465
< 
* Connection #0 to host localhost left intact
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401}

Share your .env file...
According to above

ELASTIC_PASSWORD=0123456789

And you are sure you cleaned up the volume. Just doing docker compose down does not clean up the volumes.

You have to manually remove the volumes with a docker

docker volume rm VOLUME_NAME_OR_ID

Hello,
I'm sure that I cleaned up volumes too because I issued this command

docker compose down -v

before starting over from scratch and restoring the very first password I had set up in .env.

Anyway,
today, right after rebooting my PC, it seems that everything is working fine:

$ curl -vvvv --cacert ../ca.crt -u elastic:es0000 https://localhost:9200* Host localhost:9200 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:9200...
* Connected to localhost (::1) port 9200
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: ../ca.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / X25519 / RSASSA-PSS
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=es01
*  start date: Dec 27 09:41:16 2024 GMT
*  expire date: Dec 27 09:41:16 2027 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
* Server auth using Basic with user 'elastic'
> GET / HTTP/1.1
> Host: localhost:9200
> Authorization: Basic ZWxhc3RpYzplczAwMDA=
> User-Agent: curl/8.5.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
< HTTP/1.1 200 OK
< X-elastic-product: Elasticsearch
< content-type: application/json
< content-length: 531
< 
{
  "name" : "es01",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "NdyPxfMdSiSMhc0Xg-Qf7w",
  "version" : {
    "number" : "8.7.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "f229ed3f893a515d590d0f39b05f68913e2d9b53",
    "build_date" : "2023-04-27T04:33:42.127815583Z",
    "build_snapshot" : false,
    "lucene_version" : "9.5.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
* Connection #0 to host localhost left intact

Thank you so much for your kind support!

1 Like

Glad you got it working.

Been my experience that

docker compose down -v

Is / was not always reliable at removing volumes... Maybe better now.