Authentication of [elastic] was terminated by realm [reserved]

My ELK stack version is 7.9.2

My aim is to enable authentication in Kibana login page but I'm not able to do that. I'm alays directed to the dashboard directly and it never asks for username and password

ELK servers are running on Docker containers.
In my docker-compose.yaml file elasticsearch.yml file mapping is from conf directory i.e ./elasticsearch/conf/elasticsearch.yml


and I had enabled x-pack.security: true in kibana.yml, logstash.yml and elasticserach.yml as well as docker-compose.yaml

When I tried to setup passwords by going inside the elasticserach containers and navigated to bin directory and tried running script elasticsearch-setup-passwords interactive it gave me the below error


Unexpected response code [500] from calling GET http://172.21.0.2:9200/_security/_authenticate?pretty
It doesn't look like the X-Pack security feature is enabled on this Elasticsearch node.
Please check if you have enabled X-Pack security in your elasticsearch.yml configuration file.

ERROR: X-Pack Security is disabled by configuration.

As a result I tried enabling x-pack security inside elasticsearch config folder using the below command-

xpack.security.enabled: true
xpack.monitoring.collection.enabled: true

and restarted the conatiners again- it started giving me below errors-

[INFO ][o.e.x.s.a.AuthenticationService] [inlxdev02] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

and this is my docker-compose.yaml file:

version: '3.7'
services: 
  elasticsearch: 
    build:
      context: elasticsearch/
    container_name: elasticsearch
    volumes:
      - type: bind
        source: ./elasticsearch/conf/elasticsearch.yml
        target: /usr/share/elasticsearch/conf/elasticsearch.yml
        read_only: true
      - type: volume
        source: elasticsearch
        target: /usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    environment:
      ES_JAVA_OPTS: "-Xmx2g -Xms2g"
      ELASTIC_PASSWORD: password
      ELASTIC_USERNAME : username
      # Use single node discovery in order to disable production mode and avoid bootstrap checks.
      # see: https://www.elastic.co/guide/en/elasticsearch/reference/current/bootstrap-checks.html
      discovery.type: single-node
    networks:
      - elastic
    restart: always
  
  logstash:
    container_name: logstash
    build: 
      context: logstash/
    # command: logstash -f /conf/logstash.conf
    volumes:
      - type: bind
        source: ./logstash/conf/logstash.yml
        target: /usr/share/logstash/conf/logstash.yml
        read_only: true
      - type: bind
        source: ./logstash/pipeline
        target: /usr/share/logstash/pipeline
        read_only: true
    ports:
     - "5043:5043"
     - "5044:5044"
    environment:
      LS_JAVA_OPTS: "-Xmx1g -Xms1g"
    networks:
      - elastic
    depends_on:
     - elasticsearch
    restart: always

  kibana:
    build:
      context: kibana/
    container_name: vimaan-kibana
    volumes:
      - type: bind
        source: ./kibana/conf/kibana.yml
        target: /usr/share/kibana/conf/kibana.yml
        read_only: true
    ports:
      - "5601:5601"
    networks:
        - elastic
    depends_on:
      - elasticsearch
    restart: always

volumes:
  elasticsearch:

networks:
  elastic:
    driver: bridge

Please help me @TimV @stephenb

Also- this is my Kibana UI response

Why is my elasticsearch.yml file settings not reflecting? @stephenb @TimV

Please don't post pictures of text or code or logs. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them :slight_smile:

Perhaps you should try setting this up without docker first just download the tar.gz or zip of elasticsearch and kibana and get it working the way you want. Then replicate with docker.

Also The path in docker to the elasticsearch.yml is not correct, so it is probably not being found. I think perhaps you need to be a little more careful with path, rebuilding containers etc..

Correct
/usr/share/elasticsearch/config/elasticsearch.yml
not
/usr/share/elasticsearch/conf/elasticsearch.yml

You should probably double check the other paths as well.

Kibana path is not correct either.

Logstash I don't have running...

You can use the following command to look into the containers.

docker elasticsearch -it es01 /bin/bash

@stephenb thanks a lot for your response! I was also able to figure out the root cause- path configuration.

Now, I'm able to get the login page on Kibana and everything works fine except one thing- in my elasticsearch server I'm still getting the authentication error and I'm not getting the reason-

{"type": "server", "timestamp": "2021-06-07T01:16:21,457Z", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "docker-cluster", "node.name": "fcf49d1a87d0", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "cluster.uuid": "bSnpaUuRQ0yoAhWoERU31A", "node.id": "Zj-mswZjSX-CnRKUUFkYKg"  }
{"type": "server", "timestamp": "2021-06-07T01:16:22,346Z", "level": "INFO", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "docker-cluster", "node.name": "fcf49d1a87d0", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "cluster.uuid": "bSnpaUuRQ0yoAhWoERU31A", "node.id": "Zj-mswZjSX-CnRKUUFkYKg"  }

And below is the logstash logs-

[2021-06-07T08:36:07,429][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}
[2021-06-07T08:36:07,429][WARN ][logstash.outputs.elasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elastic:xxxxxx@elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '401' contacting Elasticsearch at URL 'http://elasticsearch:9200/'"}

@warkolm I have provided the codes in proper format. The reason for posting snapshots/images were for references only and they are optional. Still I'll keep that in mind from next time. Thanks for your suggestion :slightly_smiling_face:

1 Like

401 Authentication

username and or password is incorrect

try from the command line

curl -u username:password http://localhost:9200

You will probably get the same error, if so you have not setup the passwords correct.

@stephenb Before deploying I changed the password for elastic superuser and got success message as well using elasticsearch-setup-passwords interactive

now, also I again changed it in Kibana devtool by using this command and got success as well but still getting the same authentication errors-

POST /_security/user/elastic/_password
{
"password" : "my_password"
}

@stephenb and if the username and password was wrong how I'm able to login in Kibana UI and access it?

I don't know.. but 401 is an authentication request...

You do not run the commands / steps I request so I can not help. I ask questions for a specific reason.

sorry about that. Can you help me where to run the curl command or it's equivalent request in kibana devtools?

@stephenb I ran the curl command inside elasticsearch container and got this response-

"name" : "15ff94f4yediwa",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "bcwouhwWOvsuoERU31A",
  "version" : {
    "number" : "7.9.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "mciorqu4jdk0wm3e97b4e6e",
    "build_date" : "2020-09-23T00:45:33.626720Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"

log into the logstash container and try the same curl command to check elasticsearch but you will need to use the elasticsearch container name you defined in your docker compose for elasticsearch

Log into logstash container.

curl -u user:pw http://elasticsearch:9200

I ran the curl command curl -u user:pw http://elasticsearch:9200 inside logstash conatiner and it gave this error message-

curl: (7) Failed to connect to ::1: Cannot assign requested address

@stephenb should I add xpack.security.transport.ssl.enabled: true in elasticsearch.yml file?

Is the issue Lack of SSL causing the container to not accept traffic?

Please give your suggestions.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.