Setting up Minimal Security with ElasticSearch Docker Image fails

Elastic Stack Elasticsearch
,
1

Hello everyone,

I was wondering if I could get some assistance with setting up Minimal Security (Username + Password to access the console as outlined here: Set up minimal security for Elasticsearch | Elasticsearch Guide [8.13] | Elastic) with containers.

The Dockerfile right now is simple:

FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/

Just copying over some configs as we're running this on Kubernetes. Here's our elasticsearch.yml file:

# Cluster settings
cluster.name: test-elasticsearch-cluster
network.host: 0.0.0.0

#cluster settings
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 512mb
cluster.routing.allocation.disk.watermark.high: 256mb
cluster.routing.allocation.disk.watermark.flood_stage: 128mb
discovery.type: single-node

# cluster security settings
xpack.security.enabled: false

Now I've tried a number of different things to change the password of the "elastic" user to start setting up the Minimal Security, but they don't seem to work for me for building the container. These containers are going to be ran on a Kubernetes cluster

For example, I've tried updating the Dockerfile like so from this thread:

FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/
WORKDIR /usr/share/elasticsearch/
RUN printf "elk-testing" | ./bin/elasticsearch-keystore add "bootstrap.password" -x -f

But that didn't seem to update the password as I had thought it would. I've also tried to build this Dockerfile, that I also saw from the same post, this seemed like it would accomplish what I needed but couldn't get it to complete the Docker build:

FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/
WORKDIR /usr/share/elasticsearch/
RUN printf "elk-testing" | ./bin/elasticsearch-reset-password -b -i -u elastic

I also came across this API documentation but couldn't get it to work as I need to authenticate as the "elastic" user which we haven't set up a password for yet.

Any guidance would be greatly appreciated!

2

Hi @tryin2stupelk Welcome to the community

You have turned off all security with that setting... so you can not set users and passwords.

I would review the documents here for minimal security

Also the official docker documentation here

3

Thank you for the welcome! Oh sorry about that, that's a Copy/Paste typo as I was editing, I do in fact have xpack.security.enabled: true, I'll edit the original post. Looks like I can no longer edit the original post. This is what I currently have:

# Cluster settings
cluster.name: test-elasticsearch-cluster
network.host: 0.0.0.0

#cluster settings
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 512mb
cluster.routing.allocation.disk.watermark.high: 256mb
cluster.routing.allocation.disk.watermark.flood_stage: 128mb
discovery.type: single-node

# cluster security settings
xpack.security.enabled: true

Thank you for the Documentation link! That's the same documentation I was following, but was having difficulty importing those changes into a Dockerfile to be used as a Docker Container with the Minimal Security setup.

4

Those are pretty tiny...

5

Would you recommend something like the following? We're thinking to use smaller nodes:

# Cluster settings
cluster.name: test-elasticsearch-cluster
network.host: 0.0.0.0

#cluster settings
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 4gb
cluster.routing.allocation.disk.watermark.high: 2gb
cluster.routing.allocation.disk.watermark.flood_stage: 1gb
discovery.type: single-node

# cluster security settings
xpack.security.enabled: true
6

All depends on your use case.... just be aware....

7

Does your node preserve its data directory between restarts (hopefully it does, or you'll lose all your data every time).
If it does, and you already have a password for the elastic user (which may have been auto generated) then setting bootstrap.password won't help you - security has already been bootstrapped.

It's hard to give useful advice because I don't understand your current state very well.

There are several options for configuring security on docker because docker is used for a wide variety of use cases from throw away test containers to long-running development instances, to mission critical production workloads. There will be an option that suits you, but you'll need to decide which one it is.

If you want to start a brand new container (with no existing data) with a known password for the elastic user, then you can set ELASTIC_PASSWORD to the value of the password. That's not ideal because it leaves the password in clear text in the environment variable and inside the node's keystore. But for a throwaway / dev instance that's super easy.

For more security you can do the above, and then change the password via the API after the cluster has formed.

You can run an interactive container (-i -t) and the randomly generated elastic password will be provided in the console. That's good for local manual setup, but not so good for scripted setup.

You can enter the docker container and run reset-password

You can configure a file realm user as part of your initial set up so that you have a separate superuser with a password under your control.

8

Thank you for your response Tim!

Does your node preserve its data directory between restarts

This is a great point! Right now we do have an external mount that the Pods can use to persist data, we'll have to make sure to use it for the Elastic Pods as well!

For the first option of setting ELASTIC_PASSWORD, will the following work (We're using kubernetes configmap for the value of elasticsearch.hosts: in the kibana yaml config file)?

elasticsearch Dockerfile:

# Elasticsearch Dockerfile
FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/
ENV ELASTIC_PASSWORD test-elastic-pass
ENV KIBANA_PASSWORD test-kibana-pass

elasticsearch.yaml:

# Cluster settings
cluster.name: test-elastic-cluster
network.host: 0.0.0.0

#cluster settings
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 4gb
cluster.routing.allocation.disk.watermark.high: 2gb
cluster.routing.allocation.disk.watermark.flood_stage: 1gb
discovery.type: single-node

# security settings
xpack.security.enabled: true

Kibana Dockerfile:

FROM docker.elastic.co/kibana/kibana:8.12.2
COPY --chown=1000:0 config/kibana.yml /usr/share/kibana/config/kibana.yml
RUN ./bin/kibana-keystore create
RUN printf "test-kibana-pass" | ./bin/kibana-keystore add elasticsearch.password --stdin
RUN ./bin/kibana

kibana.yml:

server.name: kibana
server.host: "0.0.0.0"
elasticsearch.hosts: [ "${ELASTICSEARCH_URL}" ]
elasticsearch.username: "kibana_system"
monitoring.ui.container.elasticsearch.enabled: true

If the above looks correct, would we access the API inside the Pod to update the password, or could we do it from outside the Pod as well?

9

I don't believe there is any option to set KIBANA_PASSWORD like that.

The docker compose example in the docs uses the API to set the Kibana password.

10

Hey Tim,

So I deployed the following Dockerfile, the same as above but removed the Kibana Password, for the Elasticsearch container:

FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/
ENV ELASTIC_PASSWORD test-elastic-pass

When I dropped into the Pod to test the password using the command curl -X GET "localhost:9200/_cluster/health?pretty" I get the following authentication error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
        "header" : {
          "WWW-Authenticate" : [
            "Basic realm=\"security\" charset=\"UTF-8\"",
            "ApiKey"
          ]
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
    "header" : {
      "WWW-Authenticate" : [
        "Basic realm=\"security\" charset=\"UTF-8\"",
        "ApiKey"
      ]
    }
  },
  "status" : 401
}

Is there another step to take with the environment variable being set in the Dockerfile to get it to use that password for elastic user?

11

Sorry, the correct command I used was: curl -X GET "localhost:9200/_cluster/health?pretty" -u elastic:test-elastic-pass passing in the password set in the Dockerfile ENV to "elastic" user which resulted in:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
        "header" : {
          "WWW-Authenticate" : [
            "Basic realm=\"security\" charset=\"UTF-8\"",
            "ApiKey"
          ]
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "unable to authenticate user [elastic] for REST request [/_cluster/health?pretty]",
    "header" : {
      "WWW-Authenticate" : [
        "Basic realm=\"security\" charset=\"UTF-8\"",
        "ApiKey"
      ]
    }
  },
  "status" : 401
}
12

I copied your dockerfile, and the yml file you posted earlier, built and ran the image and it works fine for me.

I'm not sure what you're doing differently, but what you've posted should work.

$ head -99 Dockerfile config/elasticsearch.yml
==> Dockerfile <==
FROM docker.elastic.co/elasticsearch/elasticsearch:8.12.2
COPY config/elasticsearch.yml config/
ENV ELASTIC_PASSWORD test-elastic-pass

==> config/elasticsearch.yml <==
# Cluster settings
cluster.name: test-elastic-cluster
network.host: 0.0.0.0

#cluster settings
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 4gb
cluster.routing.allocation.disk.watermark.high: 2gb
cluster.routing.allocation.disk.watermark.flood_stage: 1gb
discovery.type: single-node

# security settings
xpack.security.enabled: true

$ docker build .
...
$ docker run -p 9200:9200 -m 2GB <image-id>
...

curl -u elastic:test-elastic-pass http://localhost:9200/_security/_authenticate | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   263  100   263    0     0  15468      0 --:--:-- --:--:-- --:--:-- 16437
{
  "username": "elastic",
  "roles": [
    "superuser"
  ],
  "full_name": null,
  "email": null,
  "metadata": {
    "_reserved": true
  },
  "enabled": true,
  "authentication_realm": {
    "name": "reserved",
    "type": "reserved"
  },
  "lookup_realm": {
    "name": "reserved",
    "type": "reserved"
  },
  "authentication_type": "realm"
}
13

Thank you Tim! Looks like there might be something else in our environment preventing this from working since you were able to get it working with the same Dockerfile and config file.

Appreciate all the help! I can go ahead and close this topic as we have to do additional troubleshooting that's not related to Elastic at the moment. Thanks again!