Kibana Failed to authenticate in elasticsearch

Elastic Stack Kibana
1

I am trying to have a simple docker-compose with an elastic container and kibana.

In the logs, kibana failed to authenticate to elastic and I can't find why.

version: '3.7'
services:
  elasticsearch:
    container_name: elasticsearch
    hostname: elasticsearch
    image:     "docker.elastic.co/elasticsearch/elasticsearch:7.4.2"
    networks: ['stack']
    environment:
      - cluster.name=es-cluster
      - node.name=es-node-1
      - path.data=/usr/share/elasticsearch/data
      - http.port=9200
      - http.host=0.0.0.0
      - transport.host=127.0.0.1
      -  bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
      - xpack.security.enabled=true
      - "ELASTIC_PASSWORD=MySuperPassword"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - api_esdata1:/usr/share/elasticsearch/data
    ports:
      - '9200:9200'
    healthcheck:
      test: ["CMD", "curl","-s" ,"-f", "http://localhost:9200/_cat/health"]
  kibana:
    image: "docker.elastic.co/kibana/kibana:7.4.2"
    container_name: kibana
    environment:
      - "ELASTICSEARCH_PASSWORD=MySuperPassword"
    volumes:
      - ./kibana/config/kibana.yml:/usr/share/kibana/kibana.yml
    ports:
      - 5601:5601
    depends_on:
      - elasticsearch
    networks: ['stack']
volumes:
  api_esdata1:
    external: true
networks: {stack: {}}

and here is my ./kibana/config/kibana.yml

server.host: "0"
server.port: 127.0.0.1:5601
elasticsearch.url: "http://elasticsearch:9200"
server.name: "elastic-stack"

The elastic password works well, the problem is only on kibana, I have this error in the logs :

{"type":"log","@timestamp":"2019-11-29T09:52:53Z","tags":["status","plugin:graph@7.4.2","error"],"pid":6,"state":"red","message":"Status changed from yellow to red - [security_exception] missing authentication credentials for REST reques
t [/_nodes?filter_path=nodes..version%2Cnodes..http.publish_address%2Cnodes.*.ip], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}

Also later :

{"type":"log","@timestamp":"2019-11-29T09:52:55Z","tags":["license","warning","xpack"],"pid":6,"message":"License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. [security_exception] mi
ssing authentication credentials for REST request [/_xpack], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } } :: {"path":"/xpack","statusCode":401,"response":"{\"error\":{\"root
cause\":[{\"type\":\"security_exception\",\"reason\":\"missing authentication credentials for REST request [/_xpack]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UT
F-8\\\"\"}}],\"type\":\"security_exception\",\"reason\":\"missing authentication credentials for REST request [/_xpack]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\
\\"UTF-8\\\"\"}},\"status\":401}","wwwAuthenticateDirective":"Basic realm=\"security\" charset=\"UTF-8\""}"}

What is wrong ? Thank you

2

Hi @redfish462,

The issue I see is the ELASTIC_PASSWORD elasticsearch environment variable sets the password for the elastic user.

Kibana uses the kibana user to authenticate with Elasticsearch and would need to be setup by using the setup-passwords script: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-passwords.html

3

Thanks @mikecote . Unfortunately, I still have the error.

I tried 2 things to fix that :

  1. Set in the kibana environment "ELASTICSEARCH_USERNAME=elastic" and it's like kibana don't care about this variable

  2. I tried your solution to setup password. So I did :

Enter in the elasticsearch container to execute the script :
docker-compose exec elasticsearch bash

Launch the setup-passwords script
bin/elasticsearch-setup-passwords auto

The script returned the new passwords of service (with the kibana one). For testing, I test a curl with a new password, it works.

I changed the environnement of kibana like this so :

environment:
      - "ELASTICSEARCH_USERNAME=kibana"
      - "ELASTICSEARCH_PASSWORD=NewKibanaPassword"

Then docker-compose up -d

And I still have the error

{"statusCode":401,"error":"Unauthorized","message":"[security_exception] failed to authenticate user [kibana], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } }"}

There is a funny thing now

I have an Apache reverse Proxy before kibana with an htpassword. And I tried 3 cases :

  • Connect with Apache
    Error (the first one") : {"statusCode":401,"error":"Unauthorized","message":"[security_exception] failed to authenticate user [kibana], with { header={ WWW-Authenticate=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } }"}

  • Connect directly on the port without Apache with wrong creds
    I have the Kibana Login Page. I try elastic:changeme as passwords. I have a normal login error : Invalid username or password. Please try again.

  • Connect directly on the port without Apache with Elastic kibana Creds
    I suppose that the login is successful cause I don't have the Invalid username of password, but I have this message : {"statusCode":403,"error":"Forbidden","message":"Forbidden"}
    Here are the logs of the 403 :

Logs

{"type":"response","@timestamp":"2019-11-29T18:02:03Z","tags":,"pid":6,"method":"post","statusCode":204,"req":{"url":"/api/security/v1/login","method":"post","headers":{"host":"IP:5601","user-agent":"Mozilla/5.0 (X11; Fedora
; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"application/json, text/plain, /","accept-language":"fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate","referer":"http://IP:5601/login?next=
%2F","content-type":"application/json;charset=utf-8","kbn-version":"7.4.2","content-length":"55","dnt":"1","connection":"close"},"remoteAddress":"IP","userAgent":"IP","referer":"http://IP:5601/login?next=%2
F"},"res":{"statusCode":204,"responseTime":7,"contentLength":9},"message":"POST /api/security/v1/login 204 7ms - 9.0B"}
{"type":"response","@timestamp":"2019-11-29T18:02:03Z","tags":,"pid":6,"method":"get","statusCode":403,"req":{"url":"/","method":"get","headers":{"host":"IP:5601","user-agent":"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:60.0
) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8","accept-language":"fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3","accept-encoding":"gzip, deflate","referer":"http://IP:5601/logi
n?next=%2F","dnt":"1","connection":"close","upgrade-insecure-requests":"1"},"remoteAddress":"IP","userAgent":"IP","referer":"http://IP:5601/login?next=%2F"},"res":{"statusCode":403,"responseTime":103,"conte
ntLength":9},"message":"GET / 403 103ms - 9.0B"}

So, the question is : Why with the reverse proxy I have a 401 with no login page, and in the other hand, the direct connection on port 5601 I have a 403 on login sucessful with different error messages ?

4

If cURL works but Kibana still gives the Unauthorized error, there is probably something wrong with the docker environment variables that are used.

I would also recommend to try and get it working without the proxy first to make sure it's not causing any issues.

5

Thank you for your answer.

After some investigation :

1st problem with the reverse proxy (resolved):
The xpack security has some configurations about the headers.

**Added this in the Virtual Host fixed the problem** 
RequestHeader unset Authorization

2nd problem with {"statusCode":403,"error":"Forbidden","message":"Forbidden"} (partially resolved) :
I just tested with the elastic credentials instead of the kibana credentials and it works. I still don't know why it's not working with the kibana credentials.

6

One other way you could try debugging this is to remove the credentials from Kibana environment variables and set them within kibana.yml to see if it works there. You can set

elasticsearch.username: "kibana"
elasticsearch.password: "NewKibanaPassword"
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.