I'm struggling set up the minimal Security and the configure the TLS

Elastic Stack Elasticsearch
1

What im using:
The offical Helm Chart verion 7.17.3 from artifacthub.io
Image version is also 7.17.3

My Problem:
I'm struggling set up the minimal Security and the configure the TLS
My Steps:

  1. Create the p12
    elasticsearch@elasticsearch-master-0:~$ bin/elasticsearch-certutil cert -out -- elastic-certificates.p12 -pass ""
  2. Copy the p12 to the local computer
    kubectl cp elasticsearch-master-0:elastic-certificates.p12 elastic-certificates.p12
  3. Create a K8S Secret
    kubectl create secret generic elastic-certificates --from-file=elastic-certificates.p12
  4. Stop Elasticsearch and Kibana
    helm uninstall elasticsearch
    equal with Kibana
  5. Edit Elasticsearch valus.yaml
elasticsearch.yaml |
    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.transport.ssl.verification_mode: none 
    xpack.security.http.ssl.verification_mode: none
    xpack.security.transport.ssl.client_authentication: required
    xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    xpack.security.http.ssl.enabled: true
    xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12
    xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12 
protocol: https
secretMounts:
  - name: elastic-certificates
    secretName: elastic-certificates
    path: /usr/share/elasticsearch/config/certs
  1. Restart Elasticsearch
    helm install elasticsearch .
  2. Set up Passwords
elasticsearch@elasticsearch-master-0:~$ bin/elasticsearch-setup-passwords auto
	Changed password for user
       ...

Afterwards I get this errors

Error log from the Pod
elasticsearch-master-0: {"type": "server", "timestamp": "XXXX-XX-XXTXX:XX:XX,XXXZ", "level": "INFO", "component": "o.e.x.s.a.RealmsAuthenticator", "cluster.name": "XXXXXXXX", "node.name": "elasticsearch-master-0", "message": "Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]", "cluster.uuid": "XXXXXXXXXXX", "node.id": "XXXXXXXXXXX" }

Error log from the Cluster
Readiness probe failed: Waiting for elasticsearch cluster to become ready (request params: "wait_for_status=green&timeout=1s" ) Cluster is not yet ready (request params: "wait_for_status=green&timeout=1s" )

I think I need to set the elastic user and password some where but where and how can I set this in the Helm Chart?

2

You need to check the logs of your elasticsearch nodes, to be able to authenticate your cluster needs to be running, from the logs you shared it seems that it is not running.

The cluster does not need user/password to run, the user/password is just to authenticate users, the nodes communicate with each other using the certificates.

3

After Step 7

elasticsearch@elasticsearch-master-0:~$ bin/elasticsearch-setup-passwords auto
	Changed password for user
       ...

My pods changed from status green to yellow after Step 7. Before this step my pods were running.

4

You need to check the logs of your elasticsearch nodes.

How many data nodes you have? I do not use k8s, but a yellow status means that one of the replicas is not allocated, this does not impact you being able to log in or not, so you need to check the logs for every elasticsearch node you have.

5

How did you observe that change in status?

It might be because the probe in the helm chart uses one of the builtin users, and relies on knowing the password via environment variables. When you setup the passwords, you prevented the health probe from connecting to the cluster.

6

Thank you Tim this was the solution.
I really appreciate this advice.

The readiness probe use the Elastic user but the passwords changed. Set the new password as a k8s secret and restart elastic helped.
This was a stupid bug from my side.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.