Kibana with elastic form docker compose setup problem

habajol675 · November 24, 2023, 6:03pm

Hello everyone, I have been trying to deploy elasticsearch and kibana for several days now. I'm using docker compos and having a lot of problems. I have provided an example of the file below, but there is this error with it. It happens after I enter the user and password from kibana and try to make the settings.

[2023-11-24T17:36:46.413+00:00][ERROR][plugins.interactiveSetup.elasticsearch] Failed to authenticate with host "https://elasticsearch:9200": Hostname/IP does not match certificate's altnames: Host: elasticsearch. is not in the cert's altnames: DNS:localhost, IP Address:127.0.0.1, DNS:827c6cde4f62, IP Address:172.26.0.2

I reset all passwords via docker exec and there are no problems with this. I tried to set different settings in the environment for example, but this also did not help and caused another error.

From Elasticsearch environment:
- ELASTIC_USERNAME=elastic
- ELASTIC_PASSWORD=test123

From Kibana environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
- ELASTIC_USERNAME=kibana_system
- ELASTIC_PASSWORD=test123

If I try to use token from elasticsearch I have other error


[2023-11-24T18:14:55.752+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. socket hang up - Local: 172.28.0.3:59632, Remote: 172.28.0.2:9200

Could you please suggest a working option for docker to use elasticsearch and kibana?

All docker-compose.yaml

version: '3.4'

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.1
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - elasticsearch_data:/usr/share/elasticsearch/data
    networks:
      - elk
  
  kibana:
    image: docker.elastic.co/kibana/kibana:8.11.1
    container_name: kibana
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch
    volumes:
      - kibana_data:/usr/share/kibana/data
    networks:
      - elk

volumes:
  elasticsearch_data:
    driver: local
  kibana_data:
    driver: local

networks:
  elk:
    driver: bridge

stephenb · November 24, 2023, 6:16pm

Hi @habajol675, Welcome to the community.

Here is the official Docker Compose and Instructions....

If you just want a single node ... I created a slimmed down versions here...
it still uses the .env file

version: "3.8"

volumes:
  certs:
    driver: local
  esdata01:
    driver: local
  kibanadata:
    driver: local
  logstashdata01:
    driver: local

networks:
  default:
    name: elastic
    external: false
    
services:
  setup:
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f config/certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f config/certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 127.0.0.1\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: ["CMD-SHELL", "[ -f config/certs/es01/es01.crt ]"]
      interval: 1s
      timeout: 5s
      retries: 120

  es01:
    depends_on:
      setup:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    labels:
      co.elastic.logs/module: elasticsearch
    volumes:
      - certs:/usr/share/elasticsearch/config/certs
      - esdata01:/usr/share/elasticsearch/data
    ports:
      - ${ES_PORT}:9200
    environment:
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      # Better for Single Node
      - discovery.type=single-node
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    mem_limit: ${ES_MEM_LIMIT}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120

  kibana:
    depends_on:
      es01:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    labels:
      co.elastic.logs/module: kibana
    volumes:
      - certs:/usr/share/kibana/config/certs
      - kibanadata:/usr/share/kibana/data
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    mem_limit: ${KB_MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 5s
      timeout: 10s
      retries: 10

habajol675 · November 24, 2023, 6:37pm

Thank you very much, your solution works for me, but I’m very interested, could you tell me if I can somehow use this method without certificates and the installation section?

stephenb · November 24, 2023, 7:00pm

Just to be clear there are a number of features that require security to be configured.

If you want absolutely no security... Which you need to be very careful with as the cluster
This is fine say working on your laptop but do not put ANY sensitive data etc.

You can use this compose it turns off all security.... if you want something in between you will need to configure that yourself

you start with with
$ TAG=8.11.1 docker-compose up

You can change the memory limits , CPU etc as they are hard coded.

---
version: '3'
services:
  elasticsearch:
    container_name: es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${TAG}
    # 8.x
    environment: ['CLI_JAVA_OPTS=-Xms2g -Xmx2g','bootstrap.memory_lock=true','discovery.type=single-node','xpack.security.enabled=false', 'xpack.security.enrollment.enabled=false']
    ports:
      - 9200:9200
    networks:
      - elastic
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    deploy:    
      resources:
          limits:
            cpus: '2.0'
          reservations:
            cpus: '1.0'

  kibana:
    image: docker.elastic.co/kibana/kibana:${TAG}
    container_name: kib01
    environment:
      XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY: asdfkjhasdf-c4d3-4a0a-8290-2abcb83ab3aa
    ports:
      - 5601:5601
    networks:
      - elastic
    deploy:    
      resources:
          limits:
            cpus: '2.0'
          reservations:
            cpus: '1.0'

networks:
  elastic:

habajol675 · November 24, 2023, 7:11pm

Thank you, I have already tried this method. If I set xpack.security.enabled=false, then I can configure Kibana automatically, but there is no search, I cannot change the user and other restrictions. I want to have security such as authorization under elastic or kibana_system and also the most important thing is search to create an index and use it. The only thing I want to disable is the https check so that it does not require specifying certificates since this computer does not look outside and is only used within the network, but users and passwords should remain. Can I do as described above?

stephenb · November 24, 2023, 7:51pm

You should but you will need to do it..

I would start from the single compose I gave you and start to edit down... Take out the SSL settings and try that... If that works take out the cert Creation... You will still need to set up to set the Kibana system password and those kind of things

I'm not sure what you mean by that. You can absolutely search and use most of elasticsearch and Kibana without authentication... Did you actually try what I sent you? It works for sure I use it everyday... But yes, it does not support users and roles.

system · December 22, 2023, 7:51pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.