# How to setup X- PACK security feature for ES 7.1.0 installed from ES Helm chart **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [June 4, 2019, 1:10pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189 "2019-06-04T13:10:41Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [June 4, 2019, 1:10pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/1 "2019-06-04T13:10:41Z")
Hi All, I am trying to install elasticsearch 7 using helm on kubernetes.. Below is the helm chart of elasticsearch I am using, > > > //github.com/elastic/helm-charts/tree/main/elasticsearch However now I am able to setup the elasticsearch and kibana and its running fine. Now I want to use the security feature of x-pack in elasticsearch and kibana and I followed the below link, > > > //github.com/elastic/helm-charts/tree/main/elasticsearch I am executing "make" command and below is the error response I am getting, make > kubectl delete secrets elastic-credentials elastic-certificates elastic-certificate-pem || true && > vault read -field=value secret/devops-ci/helm-charts/elasticsearch/security/certificates | base64 --decode \> elastic-certificates.p12 && > vault read -field=value secret/devops-ci/helm-charts/elasticsearch/security/certificate-pem | base64 --decode \> elastic-certificate.pem && > kubectl create secret generic elastic-credentials --from-literal=password=changeme --from-literal=username=elastic && > kubectl create secret generic elastic-certificates --from-file=elastic-certificates.p12 && > kubectl create secret generic elastic-certificate-pem --from-file=elastic-certificate.pem && > rm -f elastic-certificates.p12 elastic-certificate.pem > Error from server (NotFound): secrets "elastic-credentials" not found > Error from server (NotFound): secrets "elastic-certificates" not found > Error from server (NotFound): secrets "elastic-certificate-pem" not found > /bin/sh: 2: vault: not found > /bin/sh: 3: vault: not found > secret "elastic-credentials" created > secret "elastic-certificates" created > secret "elastic-certificate-pem" created > helm upgrade --wait --timeout=600 --install --values ./security.yml helm-es-security ../../ ; > Error: UPGRADE FAILED: "helm-es-security" has no deployed releases > Makefile:8: recipe for target 'install' failed > make: \*\*\* [install] Error 1 Kindly help me to understand how the security feature setup done in helm charts and correct me if I am doing anything wrong. Regards, Ganeshbabu R ---
**Author:** ![michael.russell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael.russell/32/27889_2.png) [@michael.russell](https://discuss.elastic.co/u/michael.russell)\ **Post date:** [June 5, 2019, 1:02pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/2 "2019-06-05T13:02:39Z")
Hi! These examples are also part of the automated testing suite for the charts. It looks like I switched the default makefile target to include pulling the secrets from our vault service. For now can you just try running `make install` or `helm upgrade --wait --timeout=600 --install --values ./security.yml elasticsearch ../../`. That will just do the install without trying to pull in the secrets from elsewhere. I have made a note to fix up the docs as well to make sure no one else runs into this problem in the future. Thanks a lot for reporting it! ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [June 6, 2019, 6:54pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/3 "2019-06-06T18:54:19Z")
Hi @michael.russell, Thanks for the information.. I tried executing the below command, ``` ubuntu@localhost:~/KubernetesCluster/elasticsearch7/helm-charts/elasticsearch/examples/security$ helm upgrade --wait --timeout=600 --install --values ./security.yml elasticsearch ../../. Release "elasticsearch" does not exist. Installing it now. Error: release elasticsearch failed: timed out waiting for the condition ``` Below is the helm status ``` helm status elasticsearch LAST DEPLOYED: Thu Jun 6 18:33:28 2019 NAMESPACE: intell-2 STATUS: PENDING_INSTALL RESOURCES: ==> v1/Service NAME AGE security-master 9m security-master-headless 9m ==> v1beta1/StatefulSet security-master 9m ==> v1beta1/PodDisruptionBudget security-master-pdb 9m ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE security-master-0 1/2 CrashLoopBackOff 6 9m security-master-1 1/2 CrashLoopBackOff 6 9m security-master-2 1/2 CrashLoopBackOff 6 9m ==> v1/ConfigMap NAME AGE security-master-config 9m ==> v1/StorageClass elasticsearch7 9m NOTES: 1. Watch all cluster members come up. $ kubectl get pods --namespace=intell-2 -l app=security-master -w 2. Test cluster health using Helm test. $ helm test elasticsearch ubuntu@ip-localhost:~$ kubectl get pods --namespace=intell-2 -l app=security-master -w NAME READY STATUS RESTARTS AGE security-master-0 1/2 CrashLoopBackOff 6 10m security-master-1 1/2 CrashLoopBackOff 6 10m security-master-2 1/2 CrashLoopBackOff 6 10m ``` Below is the pod describe response, ``` kubectl describe pod security-master-0 -n intell-2 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 12m default-scheduler Successfully assigned intell-2/security-master-0 to ip-localhost.ec2.internal Normal SuccessfulAttachVolume 12m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-894a4-8811-11e9-82aa-0a8698864024" Normal Pulled 11m kubelet, ip-localhost.ec2.internal Container image "docker.elastic.co/elasticsearch/elasticsearch:7.1.0" already present on machine Normal Created 11m kubelet, ip-localhost.ec2.internal Created container Normal Started 11m kubelet, ip-localhost.ec2.internal Started container Normal Pulled 11m kubelet, ip-localhost.ec2.internal Container image "docker.elastic.co/elasticsearch/elasticsearch:7.1.0" already present on machine Normal Created 11m kubelet, ip-localhost.ec2.internal Created container Normal Started 11m kubelet, ip-localhost.ec2.internal Started container Normal Created 10m (x4 over 11m) kubelet, ip-localhost.ec2.internal Created container Normal Started 10m (x4 over 11m) kubelet, ip-localhost.ec2.internal Started container Normal Pulled 9m (x5 over 11m) kubelet, ip-localhost.ec2.internal Container image "docker.elastic.co/elasticsearch/elasticsearch:7.1.0" already present on machine Warning BackOff 1m (x38 over 11m) kubelet, ip-localhost.ec2.internal Back-off restarting failed container ``` Checked the logs of security-master-0 pod and found the below errors, > {"type": "server", "timestamp": "2019-06-06T18:51:49,628+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "security", "node.name": "security-master-0", "message": "uncaught exception in thread [main]" , > "stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]", > "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", > "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", > > "Caused by: java.lang.reflect.InvocationTargetException", > "at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]", > > "Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory", > "at org.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:61) ~[?:?]", > "at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]", > > "Caused by: java.io.IOException: Short read of DER length", > "at sun.security.util.DerInputStream.getLength(DerInputStream.java:588) ~[?:?]", > "at sun.security.util.DerValue.init(DerValue.java:390) ~[?:?]", > "at sun.security.util.DerValue.(DerValue.java:331) ~[?:?]", > "at sun.security.util.DerValue.(DerValue.java:344) ~[?:?]", Please correct me if I am doing anything wrong and let me know your thoughts. Regards, Ganeshbabu R ---
**Author:** ![michael.russell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael.russell/32/27889_2.png) [@michael.russell](https://discuss.elastic.co/u/michael.russell)\ **Post date:** [June 7, 2019, 10:16am UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/4 "2019-06-07T10:16:16Z")
> [@r.ganeshbabu](#): > > "Caused by: java.io.IOException: Short read of DER length", This makes it sound like the certificates wasn't generated properly or isn't in the expected format. Did you get any errors when running through the steps to generate the certificates? You could also try checking that the certificates are valid with: ```auto kubectl get secret elastic-certificates -o jsonpath='{.data.elastic-certificates\.p12}' | base64 --decode | openssl pkcs12 -info ``` ---
**Author:** ![inamatus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inamatus/32/47753_2.png) [@inamatus](https://discuss.elastic.co/u/inamatus)\ **Post date:** [June 8, 2019, 5:04pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/5 "2019-06-08T17:04:56Z")
Hey!!! This helm/charts is extemely miserble that i ever used. Any deployment can't work correctly because, the pkcs12 file doesn't work with the other pods on elastic cluster > SSL connection to [https://127.0.0.1:9200/\_security/\_authenticate?pretty](https://127.0.0.1:9200/_security/_authenticate?pretty) failed: No subject alternative names present > Please check the elasticsearch SSL settings under xpack.security.http.ssl. > ERROR: Failed to establish SSL connection to elasticsearch at [https://127.0.0.1:9200/\_security/\_authenticate?pretty](https://127.0.0.1:9200/_security/_authenticate?pretty). This shown very mistake with this helm/charts this elasticsearch can't working for any reason. Now on this time no any solution to fix this problem because, when created new CA and certificate with SANs (Subject Alternative Names) and apply on its. Not working the pod is hang and can't do anything. If delete any pod on this statefulset the IP Address was change and certificate can't used for new pods. Elastic Cloud on K8S can working completely but it's an alpha version . It had many limitation for using in this time (can't customize with dedicated node). ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [June 11, 2019, 1:34pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/6 "2019-06-11T13:34:13Z")
> [@michael.russell](#): > > This makes it sound like the certificates wasn't generated properly or isn't in the expected format. Did you get any errors when running through the steps to generate the certificates? Hi @michael.russell I have a doubt in generating the certificates, but the official docs saying use the `elasticsearch-certutil ca` command to generate ca & certificate file > bin/elasticsearch-certutil ca > bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 How should i generate the certificates ( **elastic-certificates.p12** ) before I executing the kubernetes secrets for authentication, **Create Kubernetes secrets for authentication credentials and certificates** ```auto kubectl create secret generic elastic-credentials --from-literal=password=changeme --from-literal=username=elastic kubectl create secret generic elastic-certificates --from-file=elastic-certificates.p12 ``` Do I need to use openssl for generating node certificates? Kindly share your thoughts.. Regards, Ganeshbabu R ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [June 14, 2019, 7:42am UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/7 "2019-06-14T07:42:18Z")
Hi @michael.russell, I tried generating certificates using the openssl command, below is the steps I tried, ``` ubuntu@ip-172-16-27-109:~/KubernetesCluster/elasticsearch7/helm-charts$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /home/ubuntu/KubernetesCluster/elasticsearch7/helm-charts/elastic.key -out /home/ubuntu/KubernetesCluster/elasticsearch7/helm-charts/elastic.crt Generating a 2048 bit RSA private key .....................................+++ ...........+++ writing new private key to '/home/ubuntu/KubernetesCluster/elasticsearch7/helm-charts/elastic.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:IN State or Province Name (full name) [Some-State]:Bangalore Locality Name (eg, city) []:Karnataka Organization Name (eg, company) [Internet Widgits Pty Ltd]:Microland Organizational Unit Name (eg, section) []:Automation Common Name (e.g. server FQDN or YOUR name) []:security-master Email Address []:abc@microland.com ``` I gave the Common Name as **"security-master"** because the name of elasticsearch service as same then I tried to converting to .p12 format by using the below steps, 1. I copied both key & crt files into the txt file and execute the below command `openssl pkcs12 -export -in elastic.txt -out elastic-certificates.p12` Now I have created Kubernetes secrets for authentication credentials and certificates and then executed the below helm command to setup elasticsearch with security enabled, > helm upgrade --namespace intelligeni-2 --wait --timeout=600 --install --values /home/ubuntu/KubernetesCluster/elasticsearch7/helm-charts/elasticsearch/examples/security/security.yml elasticsearch /home/ubuntu/KubernetesCluster/elasticsearch7/helm-charts/elasticsearch Below is the **helm status elasticsearch** ``` ubuntu@ip-172-16-27-109:~$ helm status elasticsearch LAST DEPLOYED: Wed Jun 12 09:57:59 2019 NAMESPACE: intelligeni-2 STATUS: DEPLOYED RESOURCES: ==> v1beta1/PodDisruptionBudget NAME AGE security-master-pdb 1d ==> v1/Pod(related) NAME READY STATUS RESTARTS AGE security-master-0 2/2 Running 0 1d security-master-1 2/2 Running 0 1d security-master-2 2/2 Running 0 1d ==> v1/ConfigMap NAME AGE security-master-config 1d ==> v1/StorageClass elasticsearch7 1d ==> v1/Service security-master 1d security-master-headless 1d ==> v1beta1/StatefulSet security-master 1d NOTES: 1. Watch all cluster members come up. $ kubectl get pods --namespace=intelligeni-2 -l app=security-master -w 2. Test cluster health using Helm test. $ helm test elasticsearch ``` I went inside one of the elasticsearch pod **kubectl exec -it security-master-0 bash -n** and to check the cluster health with the authentication, ``` [root@security-master-0 elasticsearch]# curl -u elastic:changeme http://security-master:9200/_cluster/health?pretty=true curl: (52) Empty reply from server [root@security-master-0 elasticsearch]# curl -u elastic:changeme http://localhost:9200 curl: (52) Empty reply from server ``` **Can you please tell me why the response is showing as empty reply from server?** ``` [root@security-master-0 elasticsearch]# curl -u elastic:changeme https://security-master:9200/_cluster/health?pretty=true curl: (60) Peer's certificate issuer has been marked as not trusted by the user. More details here: http://curl.haxx.se/docs/sslcerts.html curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. ``` **If I change the protocol to https I am getting the above response but now sure why certificate verification is failed?** ``` [root@security-master-0 elasticsearch]# curl -k -u elastic:changeme https://security-master:9200/_cluster/health?pretty=true { "cluster_name" : "security", "status" : "green", "timed_out" : false, "number_of_nodes" : 3, "number_of_data_nodes" : 3, "active_primary_shards" : 3, "active_shards" : 6, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } ``` **But if i add -k or --insecure I can able to verify the cluster health with the authentication but still I think certificate verification is not doing properly. Is it because of self signed certificate creation?** Please correct me if I am doing anything wrong and let me know your thoughts. Thanks, Ganeshbabu R ---
**Author:** ![michael.russell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael.russell/32/27889_2.png) [@michael.russell](https://discuss.elastic.co/u/michael.russell)\ **Post date:** [June 20, 2019, 3:22pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/8 "2019-06-20T15:22:40Z")
> **But if i add -k or --insecure I can able to verify the cluster health with the authentication but still I think certificate verification is not doing properly. Is it because of self signed certificate creation?** Nope, it looks like this is working perfectly! You are getting a warning about it being a self signed certificate because it is a self signed certificate :). To get curl to work without ignoring certificate warnings you will need to run `curl --cacert /path/to/your/self/signed/ca.whatever`. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 18, 2019, 3:22pm UTC](https://discuss.elastic.co/t/how-to-setup-x-pack-security-feature-for-es-7-1-0-installed-from-es-helm-chart/184189/9 "2019-07-18T15:22:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.