# Testing TLS/SSL secured elastic cluster **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [September 30, 2020, 10:25pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572 "2020-09-30T22:25:41Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [September 30, 2020, 10:25pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/1 "2020-09-30T22:25:41Z")
I configured 2 node es cluster with the TLS security section in es yml file using certificates in PKCS#12 format, native realm by default, basic license, elasticsearch-7.8.0-linux-86\_64.tar.gz, RHEL 5.11. Used command with no password: bin/elasticsearch-certutil cert -out config/certs/elastic-certificates.p12 -pass "" I placed generated file elastic-certificates.p12 on 2 nodes in config/certs/, so it is not host/node specific. Elasticsearch.yml > ..... > xpack.security.enabled: true > #TLS > xpack.security.transport.ssl.enabled: true > xpack.security.transport.ssl.verification\_mode: certificate > xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 > xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 > #SSL > xpack.security.transport.ssl.enabled: true > xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12 > xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 > xpack.security.http.ssl.client\_authentication: optional > xpack.security.transport.ssl.supported\_protocols: ["TLSv1.2", "TLSv1.1", "TLSv1"] Created custom role, assigned privileges to it; created custom user estester with estester\_pwd, assigned the role to the user; **Testing TLS only use case** - With the commented out the SSL section in config file (testing TLS only) I was able to successfully test the cluster with the curl command like `curl -u estester:estester_pwd -XGET "http://host:port/_cluster/health?pretty"` **Testing TLS/SSL use case** - After then I uncommented the SSL section in config file (to test SSL), restarted nodes Ran > curl -u estester:estester\_pwd -XGET "http **s** ://host:port/\_cluster/health?pretty" **Es log:** > ... io.netty..handler.codec..DecoderException: javax.net.ssl.SSLException: Received fatal alert: uncnown.ca **Response on console** : > ``` > SSL certificate problem, verify that the CA cert is ok. Details: > error : SSL routines: SSL3_GET_SERVER_CERTIFICATE:certificate verify failed > curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). The default bundle is named curl-ca-bundle.crt; you can specify an alternate file using the --cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. > > ``` Any advise how this might be fixed ? What curl command might be used to test SSL on the cluster in my use case? Is there any command to convert passwordless elastic-certificates.p12 from PKCS#12 into PEM format ? Thanks in advance ---
**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\ **Post date:** [October 1, 2020, 5:13am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/2 "2020-10-01T05:13:53Z")
Hi, please format your codes and errors under preformatted text `` or `backticks (```) `as it is hard to read otherwise. 1. When you start the service, did you see any errors in log? 2. To test certs using curl, you can use ` curl --cacert /path/to/ca.crt -u elastic 'https://xxx:9200/_cat/nodes?v'` 3. To convert p12 to pem, you can use `openssl` for example see link below [Make .p12 to .pem for kibana](https://discuss.elastic.co/t/make-p12-to-pem-for-kibana/183231/6) Do share the outcome from 1 and 2. ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 1, 2020, 1:49pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/3 "2020-10-01T13:49:15Z")
Thank you for the response Kavier 1 - When I start nodes I can see in logs there were no errors; nodes started successfully, status of nodes are GREEN 2 - To test the cluster with the curl I need ca.crt which I don't have. As I mentioned initially I generated passwordless (with no password provided) elastic-certificates.p12 as follows `bin/elasticsearch-certutil cert -out config/certs/elastic-certificates.p12 -pass ""` To convert it to PEM I ran command: `openssl pkcs12 -in certs/elastic-certificates.p12 -out certs/elastic-certificates.crt -nokeys` I has been prompted for password, clicked enter (because I have no password) and empty file elastic-certificates.crt has been created with zero size. Q: how I can create ca.crt from my passwordless elastic-certificates.p12 ? ---
**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\ **Post date:** [October 1, 2020, 2:11pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/4 "2020-10-01T14:11:17Z")
> [@dzyubanv](#): > > openssl pkcs12 -in certs/elastic-certificates.p12 -out certs/elastic-certificates.crt -nokeys I'm not an expert for certs, but I wonder will it be different if you -out cert **.pem** instead? ```auto openssl pkcs12 -in elastic-certificates.p12 -out elastic-ca.pem -nokeys ``` I always use below cheatsheet when dealing with certs issue.. hope it helps. [https://www.sslshopper.com/article-most-common-openssl-commands.html](https://www.sslshopper.com/article-most-common-openssl-commands.html) ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 1, 2020, 3:00pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/5 "2020-10-01T15:00:08Z")
Created .crt file: `openssl pkcs12 -in certs/elastic-certificates.p12 -out certs/elastic-certificates.crt -cacerts -nokeys` Ran request: `curl --cacert /path/to/elastic-certificates.crt -u elastic 'https://xx.xxx.xx.xxx:9200/_cat/nodes?v'` Response on console: `curl: (51) SSL: certificate subject name 'instance' does not match target host name 'xx.xxx.xx.xxx'` Elastic log: nothing changed after running the request ---
**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\ **Post date:** [October 1, 2020, 3:18pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/6 "2020-10-01T15:18:12Z")
Can you run below to check on the cert? `openssl x509 -in certificate.crt -text -noout ` ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 1, 2020, 3:34pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/7 "2020-10-01T15:34:31Z")
Yes, ran the command successfully and can see the output on the console. I can't share it because I'm working on corp environment and chatting here from my personal laptop. The content of file looks good, it contains two sections - Subject public key ... and X509v3 Basic constrains :critical CA:TRUE ... The word "instance" is not in there ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 1, 2020, 4:38pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/8 "2020-10-01T16:38:42Z")
I think the response on console `curl: (51) SSL: certificate subject name 'instance' does not match target host name 'xx.xxx.xx.xxx'` means the client (curl) tries to match CN name 'instance' with the domain name in URL request. The question is why it takes the 'instance' and where the 'instance' come from ? ---
**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\ **Post date:** [October 1, 2020, 4:42pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/9 "2020-10-01T16:42:12Z")
Looking back at the very first begining, > [@dzyubanv](#): > > bin/elasticsearch-certutil cert -out config/certs/elastic-certificates.p12 -pass "" This cert is generated without CA > [@dzyubanv](#): > > ... io.netty..handler.codec..DecoderException: javax.net.ssl.SSLException: Received fatal alert: uncnown.ca and this line is saying it cant find a CA Is it possible for you to retry the SSL TLS setup following below document? 1. Generate CA / CSR to sign by CA 2. Generate node cert with above CA 3. Move cert to ideal directory 4. Put the cert config into elasticsearch.yml > **[Configure TLS | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates)** ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 2, 2020, 12:54am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/10 "2020-10-02T00:54:34Z")
Kavier, repeated the steps once again as you advised according the doc In step 3 of es doc when generated elastic-certificates.crt, was prompted for host names, ips of hosts (I entered that but I don't want to use them) Created .crt file as follows: `openssl pkcs12 -in certs/elastic-certificates.p12 -out certs/elastic-certificates.crt -cacerts -nokeys` Restarted nodes Test/request the TLS/SSL secured cluster: `curl --cacert /path/to/elastic-certificates.crt -u elastic 'https://xx.xxx.xx.xxx:9200/_cat/nodes?v'` Response on console: `curl: (51) SSL: certificate subject name 'instance' does not match target host name 'xx.xxx.xx.xxx'` Elastic log: no new lines appeared in the log after running the request **Issue: wrong response** Am I missing something (?) Also, my elasticsearch.yml contains the lines `xpack.security.transport.ssl.verification_mode: certificate` `xpack.security.http.ssl.client_authentication: optional` and according to the doc the handshake process should not perform verification of matching the host/subject from the certificates against the host in URL, correct ? When testing the cluster with curl, should I use the certificate file created by `openssl pkcs12 -in certs/elastic-certificates.p12 -out certs/elastic-certificates.crt -cacerts -nokeys` instead of certificate file created in step 3 of es doc ? ---
**Author:** ![kavierkoo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kavierkoo/32/86555_2.png) [@kavierkoo](https://discuss.elastic.co/u/kavierkoo)\ **Post date:** [October 2, 2020, 3:47am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/11 "2020-10-02T03:47:37Z")
@warkolm Could you shed some light on this issue? Thanks!! ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [October 6, 2020, 9:19am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/12 "2020-10-06T09:19:38Z")
The original problem in your first post is that you have configured elasticsearch with a certificate that is signed by a custom CA so your operating system ( and `curl` by extension) doesn't know that it should trust it. You attempt to solve this by telling `curl` explicitly that it should trust this CA. ( Note that this should solve your problem for _your_ machine and curl but not for anyone else ) Lets take things one by one. - > Testing TLS only use case\*\* ..... > Testing TLS/SSL use case\*\* TLS and SSL are the same thing. TLS is a newer version of SSL and terms are used in literature interchangeably ( unfortunately ) . - `xpack.security.transport.ssl.verification_mode: certificate`: This is not a secure configuration. Any reason in specific that you want to disable hostname verification? - The latest problem you have is that you have created a certificate for a node and you tell it that the hostname to which it applies is `X`. Then, you try to access that node by `Y` . And your local tool ( curl ) tells you that the certificate is for `X` and not for `Y` and so, it can't validate it. You need to recreate the certificate and add the correct Subject Alternate Names ( with `--dns` and `--ip` ) - > was prompted for host names, ips of hosts (I entered that but I don't want to use them) If you add the IP addresses as SANs then you can remove the `xpack.security.transport.ssl.verification_mode: certificate` setting too and make your configuration much more secure. HTH ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 6, 2020, 9:17pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/13 "2020-10-06T21:17:34Z")
Thanks for your attention Ioannis, I understand your responses but still have some questions I'm trying to setup secured es cluster as follows: (a)setup/config RBAC for particular user/password test with curl (b)setup/config TLS/SSL secure inter-node communication with one single certificate which might be used for every node in the cluster test with curl (c)setup/config TLS/SSL for HTTP client, external communication - meaning java app/client will request the es secured cluster via http **s** test with curl (d)test secured es cluster with java/app client via http **s** Also, no Kibana install/setup, all setup/config are done via es REST API Currently I'm working on ES-7.8 poc cluster (RHEL 5.11) with two hosts and two nodes on each host (one is master, one is data), but real life cluster will contain up to 20 hosts and up to 20 nodes on each with the ability to add more hosts/nodes if needed. (a)initially I setup RBAC on unsecured cluster for user:estester and pwd: estester\_pwd with privileges "monitor" cluster and "all" for indices; tested this with curl successfully (b)after then I setup/config TLS/SSL secure inter-node communication without hosts/ips/SANs in the certificate as described in steps 1 and 2 at es doc [Encrypting communications in Elasticsearchedit](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/configuring-tls.html#node-certificates) and generated certificate in step 2 without SAN in it. I tested the cluster successfully with the "curl -k ..." and the elasticsearch.yml below elasticsearch.yml ... cluster name, node name, etc, ... xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification\_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.enabled: true xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.client\_authentication: optional xpack.security.transport.ssl.supported\_protocols: ["TLSv1.2", "TLSv1.1", "TLSv1"] **The reason for not using hostnames/IPs verification in the certificate for inter-node communication** is the real life cluster will be running on hosts protected by corp firewall and will not be exposed for public external internet communication. **Another reason** is to simplify the process of adding new hosts/nodes if needed and use the same certificate for inter-node communication **Q: is this setup/config without hosts/IPs in the certificate possible/reasonable taking into account having corp firewall on all hosts ?** **Q: to my understanding this certificate can Not be used for external communication from app/java client to the cluster, correct ?** (c)then I generated certificate for external HTTP client communication, step 3 (optional) at es doc When prompted for "Generate certificate per node? [y/N]" I selected the "y" and when prompted, provided node name and host name for the first host, first node, etc. Unzipped generated zip file, placed http.p12 in the config dir of the first node. Changed elasticsearch.yml per es doc as follows: xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification\_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.enabled: true **xpack.security.http.ssl.keystore.path: "http.p12"** xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.client\_authentication: optional xpack.security.transport.ssl.supported\_protocols: ["TLSv1.2", "TLSv1.1", "TLSv1"] Restarted nodes. Converted http.p12 to http.crt `openssl pkcs12 -in http.p12 -out http.crt -cacerts -nokeys` Tested cluster successfully with expected response making request on the host, which are in the certificate, as follows `curl --cacert /path/to/http.crt -u estester:estester_pwd 'https://xx.xxx.xx.xxx:9200/_cat/nodes?v'` I still have some questions related to (c) above please **Q: when prompted with question "Generate certificate per node? [y/N]" and if I wish to generate single certificate that is valid across all the hostnames or ip addresses in the cluster, should I go with the "y" or "N" ?** **Q: should I change the xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 for xpack.security.http.ssl.truststore.path: "http.p12" in the above .yml** **Q: with question "Generate certificate per node? [y/N]" (y or N) when prompted for node name, host name should I enter all node names (in my use case 4) and all host names (in my use case 2) OR just master nodes and hosts which will be used by java app client to request the cluster via https ?** Having all these final and correct setup/config in place, will allow to go with (d) testing es secured cluster with java/app client Thanks a lot in advance ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [October 7, 2020, 5:08am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/14 "2020-10-07T05:08:33Z")
> [@dzyubanv](#): > > ```auto > xpack.security.enabled: true > xpack.security.transport.ssl.enabled: true > xpack.security.transport.ssl.verification_mode: certificate > xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 > xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 > xpack.security.transport.ssl.enabled: true > xpack.security.http.ssl.keystore.path: "http.p12" > xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 > xpack.security.http.ssl.client_authentication: optional > xpack.security.transport.ssl.supported_protocols: ["TLSv1.2", "TLSv1.1", "TLSv1"] > > ``` In this config (and earlier in this thread) you have ```auto xpack.security.transport.ssl.enabled: true ``` twice, but do not have ```auto xpack.security.http.ssl.enabled: true ``` If you want the node to be accessible over `https`, then you must enable `http.ssl` > [@dzyubanv](#): > > Q: when prompted with question "Generate certificate per node? [y/N]" and if I wish to generate single certificate that is valid across all the hostnames or ip addresses in the cluster, should I go with the "y" or "N" ? If you want one single certificate for the whole cluster, then you do not want a _certificate per node_ and should answer **N** to this question. > [@dzyubanv](#): > > Q: should I change the xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 for xpack.security.http.ssl.truststore.path: "http.p12" in the above .yml Yes, your `certs/elastic-certificates.p12` file is intended to secure the `transport` protocol that the nodes use to connect to each other. For http you should use the `http.p12` you generated. > [@dzyubanv](#): > > Q: with question "Generate certificate per node? [y/N]" (y or N) when prompted for node name, host name should I enter all node names (in my use case 4) and all host names (in my use case 2) OR just master nodes and hosts which will be used by java app client to request the cluster via https ? You should enter every hostname/IP that any client (including curl, or administrative tools) will ever use to connect to your cluster. If in doubt, that should be every node. > just master nodes and hosts which will be used by java app client to request the cluster via https It is not recommended to have clients connect directly to master nodes. If your cluster has dedicated master nodes, then they should be _dedicated_ to being master nodes, and should not handle client requests. ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 7, 2020, 10:32pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/15 "2020-10-07T22:32:10Z")
Thanks for the quick response Tim. That twice repeated line in configuration yml file is just typo error here, apology for the inconvenience. In the real cluster it was the correct line. elasticsearch.yml ``` ... cluster name, node name, etc, ... # Security configuration xpack.security.enabled: true # TLS/SSL encryption inter-node communication xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 # TLS/SSL encryption http client communication xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: "http.p12" xpack.security.http.ssl.truststore.path: "http.p12" xpack.security.http.ssl.client_authentication: optional xpack.security.transport.ssl.supported_protocols: ["TLSv1.2", "TLSv1.1", "TLSv1"] ``` I re-generated certificates for inter-node and http client communication and was able to test different use cases with the curl and authorized user as follows ``` curl --cacert /path/to/http.crt -u estester:estester_pwd -XGET 'https://ip:port/_cat/nodes?v' curl --cacert /path/to/http.crt -u estester:estester_pwd -XGET 'https://host:port/_cat/nodes?v' curl -u estester:estester_pwd -XGET 'https://host:port/_cat/nodes?v' curl --cacert /path/to/http.crt -XGET 'https://host:port/_cat/nodes?v' ``` etc. Thank you all - Kavier, Ioannis, Albert Zaharovits and Tim Vernum - for your patience, help and generosity. Now, when the x-pack secured cluster is configured and tested with the curl, I'm going to implement java app to test it with elastic cluster. If you can reference me to simple java code samples to test secured elastic cluster with the client certificate and authorized user via https, it will be appreciated. ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 16, 2020, 1:45am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/16 "2020-10-16T01:45:59Z")
**Running java client on Linux gives ClassNotFoundException** I implemented simple java client based on JEST API Project contains jest-6.3.1.jar and x-pack-transport-7.8.1.jar Compiled java code successfully, ran it on Linux and got the exception Details are below ``` import java.io.InputStream; import java.nio.file.Files; import java.nio.file.Path; import java.security.KeyStore; import java.security.cert.Certificate; import java.security.cert.CertificateFacrory; import java.util.HashMap; import java.util.Map; import javax.net.ssl.HostnameVerifier; import javax.net.ssl.SSLContext; import org.apache.http.conn.ssl.NoopHostnameVerifier; import org.apache.http.conn.ssl.SSLConnectionSocketFactory; import org.apache.http.nio.conn.SchemeIOSessionStrategy; import org.apache.http.nio.conn.SSLIOSessionStrategy; import org.apache.http.ssl.SSLContextBuilder; import org.apache.http.ssl.SSLContexts; import com.google.gson.JsonObject; import io.searchbox.client.JestClient; import io.searchbox.client.JestClientFactory; import io.searchbox.client.JestResult; import io.searchbox.client.config.HttpClientConfig; import io.searchbox.core.Search; public class JestConnSecurES { public static void main() throws Exception { String host=args[0]; String port=args[1]; String index=args[2]; String path_to_ca_crt=args[3]; String accessToken=args[4]; Path trustStorePath = Paths.get("/path/to/truststore.p12"); KeyStore truststore = KeyStore.getInstance("pkcs12"); try (InputStream is = Files.newInputStream(trustStorePath)) { truststore.load(is, keyStorePass.toCharArray()); } SSLContextBuilder sslBuilder = SSLContexts.custom() .loadTrustMaterial(truststore, null); final SSLContext sslContext = sslBuilder.build(); //CA certificate is available as PEM encoded file *.crt Path caCrtificatePath=Path.get(path_to_ca_crt); CertificateFactory certFactory= CertificateFactory.getInstance("X.509"); Certificate trustedCa; try (InputStream is=Files.newInputStream(caCrtificatePath=Path)){ trustedCa=certFactory.generateCertificate(is); } KeyStore trustStore=keyStore.getInstance("pkcs12"); trustStore.load(null,null); trustStore.setCertificateEntry("ca", trustedCa); SSLContextBuilder sslContextBuilder=SSLContexts.custom() .loadTrustMaterial(trustStore, null); final SSLContext sslContext=sslContextBuilder.build(); HostnameVerifier hostnameVerifier=NoopHostnameVerifier.INSTANCE; SSLConnectionSocketFactory sslSocketFactory= new SSLConnectionSocketFactory(sslContext, hostnameVerifier); SchemeIOSessionStrategy httpsIOSessionStrategy= new SSLIOSessionStrategy(sslContext, hostnameVerifier); JestClientFactory factory=new JestClientFactory(); factory.setHTTPClientConfig(new HttpClientConfig.Builder("https://"+host+":"+port) .defaultSchemeForDiscoveredNodes("https") .sslSocketFactory(sslSocketFactory) .httpsIOSessionStrategy(httpsIOSessionStrategy) .build() ); JestClient client=factory.getObject(); String query=""; Search search=null; Map header=new HashMap<>; header.put("Authorization", "Basic "+accessToken); search=new Search.Builder(query).addIndex.setHeader(header).build(); JestResultn result=client.execute(search); JsonObject jo=result.getJsonObject(); System.out.println("search result \n "+jo.toString()); } } ``` **Running java code on Linux** java -jar [java\_main\_class] [host] [port] [index] [path\_to\_ca\_crt] [accessToken] **Console output** ``` Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/conn/socket/LayeredConnectionSocketFactory .......................................................................................................... Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.socket.LayeredConnectionSocketFactory ``` Am I missing some specific dependency, jar(s) ? Any help will be appreciated ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [October 17, 2020, 2:44pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/17 "2020-10-17T14:44:25Z")
The code you shared has nothing Elasticsearch specific in it, so it's hard for us to troubleshoot what might be wrong. Instead of treating ES as a generic REST API that you interact over HTTP , you can also use the High Level Rest Client, details are available in our docs: [https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high.html) ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 17, 2020, 5:17pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/18 "2020-10-17T17:17:49Z")
Thanks for the response Ioannis In the shared code I tried to achieve: [1]using JEST API, because there are lots of legacy code written on it for java client, currently communicating with ES-2.3/ES-5.6 clusters with ReadOnlyRest plugin and simplify the migration to newest ES-7.8 [2]use X-Pack security instead of ROR plugin for java client to communicate with secured ES-7.8 **Q1: In your opinion, is it possible to use JEST API to communicate with X-Pack secured ES-7.8 cluster or the only High or Low Level Rest Clients should be used without mixing with JEST API ?** **Q2: Which one is more preferable or must be used for java client implementation - High or Low Level Rest Client ?** Some explanation to shared code and issues: ``` public class JestConnSecurES { public static void main() throws Exception { String host=args[0]; String port=args[1]; String index=args[2]; String path_to_ca_crt=args[3]; String accessToken=args[4]; //CA certificate is available as PEM encoded file *.crt Path caCrtificatePath=Path.get(path_to_ca_crt); CertificateFactory certFactory=CertificateFactory.getInstance("X.509"); Certificate trustedCa; try (InputStream is=Files.newInputStream(caCrtificatePath=Path)){ trustedCa=certFactory.generateCertificate(is); } KeyStore trustStore=keyStore.getInstance("pkcs12"); trustStore.load(null,null); trustStore.setCertificateEntry("ca", trustedCa); SSLContextBuilder sslContextBuilder=SSLContexts.custom() .loadTrustMaterial(trustStore, null); final SSLContext sslContext=sslContextBuilder.build(); HostnameVerifier hostnameVerifier=NoopHostnameVerifier.INSTANCE; SSLConnectionSocketFactory sslSocketFactory= new SSLConnectionSocketFactory(sslContext, hostnameVerifier); SchemeIOSessionStrategy httpsIOSessionStrategy= new SSLIOSessionStrategy(sslContext, hostnameVerifier); JestClientFactory factory=new JestClientFactory(); factory.setHTTPClientConfig(new httpClientConfig.Builder("https://"+host+":"+port) .defaultSchemeForDiscoveredNodes("https") .sslSocketFactory(sslSocketFactory) .httpsIOSessionStrategy(httpsIOSessionStrategy) .build() ); JestClient client=factory.getObject(); String query=""; Search search=null; Map header=new HashMap<>; header.put("Authorization", "Basic "+accessToken); search=new Search.Builder(query).addIndex.setHeader(header).build(); JestResultn result=client.execute(search); JsonObject jo=result.getJsonObject(); System.out.println("search result \n "+jo.toString()); } } ``` [3]java client project in Eclipse/pom.xml references jest-6.3.1.jar and x-pack-transport-7.8.1.jar as advised at [https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-xpack-client.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-xpack-client.html) [4]first section of code from "Path trustStorePath = " till "final SSLContext sslContext=" I got as it is at [https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/\_encrypted\_communication.html](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/_encrypted_communication.html) because it allows to use the certificate file \*.crt I tested with curl successfully on the cluster In this section the "org.apache.http.conn.ssl.SSLContextBuilder" and "org.apache.http.conn.ssl.SSLContexts" say "SSLContextBuilder and SSLContexts deprecated" **Q3: Which packages/jars do contain these deprecated SSLContextBuilder and SSLContexts ?** [5]section of code from "JestClient client=factory.getObject()" till "JsonObject jo=result.getJsonObject()" has been tested successfully previously with ES-5.6/ROR plugin cluster [6]shared java code constructs trustStorePath, sslBuilder, sslContext, caCrtificatePath, trustStore, sslContextBuilder, sslContext, sslSocketFactory, factory, client, header, search, result with requested index The code accepts input arguments: host, port, index to search, path to certificate **Q4: based on [3]-[6] I'm not clear why you're saying "code you shared has nothing Elasticsearch specific in it" ?** I might be missing something, having not much experience in the field I hope having answers to Q1-Q4 and references to practical examples can help me to proceed further in right direction with java client implementation. Also will follow up your advise to look at High Level Rest Client Many thanks in advance ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [October 18, 2020, 9:15am UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/19 "2020-10-18T09:15:58Z")
Hi, Apologies for the previous post, I didn't mean to put you off - sorry if it came out this way. > [@dzyubanv](#): > > Q1: In your opinion, is it possible to use JEST API to communicate with X-Pack secured ES-7.8 cluster or the only High or Low Level Rest Clients should be used without mixing with JEST API ? I have no working knowledge of this JEST API that you mention so I can't state an opinion. I know for a fact that {H,L}LRC work fine and that the transport client is deprecated in 7 and will be removed in 8 so my suggestion is to use the rest client. > [@dzyubanv](#): > > Q2: Which one is more preferable or must be used for java client implementation - High or Low Level Rest Client ? I would use the HLRC unless you know why you have a specific reason to use the LLRC instead. > [@dzyubanv](#): > > **Q3: Which packages/jars do contain these deprecated SSLContextBuilder and SSLContexts ?** It's in the apache http client. > [@dzyubanv](#): > > Q4: based on [3]-[6] I'm not clear why you're saying "code you shared has nothing Elasticsearch specific in it" ? Look at your imports. You are _not_ using anything from the elasticsearch transport client in your code, and there is nothing elasticsearch related in your code. You use some libraries to make a call to an HTTP REST API ( that, ok, happens to be elasticsearch in this case , but this is not relevant in that context ) > [@dzyubanv](#): > > [3]java client project in Eclipse/pom.xml references jest-6.3.1.jar and x-pack-transport-7.8.1.jar as advised at [https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-xpack-client.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-xpack-client.html) > [4]first section of code from "Path trustStorePath = " till "final SSLContext sslContext=" I got as it is at [Encrypted communication | Java REST Client [7.9] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.9/_encrypted_communication.html) > because it allows to use the certificate file \*.crt I tested with curl successfully on the cluster > In this section the "org.apache.http.conn.ssl.SSLContextBuilder" and "org.apache.http.conn.ssl.SSLContexts" say "SSLContextBuilder and SSLContexts deprecated" In 3 you just added the xpack client as a dependency but your are _not_ using it anywhere. In 4 you are copying part of an example from a LLRC doc and attempt to apply it to a transport client setup, which won't work. > [@dzyubanv](#): > > I hope having answers to Q1-Q4 and references to practical examples It's really hard for folks to provide generic consulting on development efforts in such kind of forums. I'd start by reading the rest client docs that were linked above. They have all the necessary setup and they contain examples and sections to get you going. HTH ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [October 18, 2020, 2:40pm UTC](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572/20 "2020-10-18T14:40:39Z")
Thank you Ioannis for your attention and detailed answers, will read HLRC docs and give it a try. [Next page](https://discuss.elastic.co/t/testing-tls-ssl-secured-elastic-cluster/250572.md?page=2)