# SSL Certificate error when connecting to elastic search from python **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [May 28, 2019, 3:58am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977 "2019-05-28T03:58:10Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![udara](https://avatars.discourse-cdn.com/v4/letter/u/65b543/32.png) [@udara](https://discuss.elastic.co/u/udara)\ **Post date:** [May 28, 2019, 3:58am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/1 "2019-05-28T03:58:11Z")
I have set up the elastic search 7.1 in my local server under port 9650. And Signed with a self-generated CA. We have connected kibana using the steps mentioned in the below documentation. [https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates) Below is the Kibana.yml configuration. ``` elasticsearch.username: "elastic" elasticsearch.password: "testpassword" elasticsearch.ssl.certificate: /etc/kibana/config/client.cer elasticsearch.ssl.key: /etc/kibana/config/client.key elasticsearch.ssl.certificateAuthorities: /etc/kibana/config/client-ca.cer elasticsearch.ssl.verificationMode: certificate ``` I was able to connect to ES from kibana successfully. Now when I use the same certificates to connect to ES from python I'm getting the below error in the python code. `ConnectionError(HTTPSConnectionPool(host='10.8.1.205', port=9650): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))) caused by: SSLError(HTTPSConnectionPool(host='10.8.1.205', port=9650): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),)))` and the below error from Elastic search log. ``` moteAddress=/10.8.1.205:58122} io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final] at java.lang.Thread.run(Thread.java:835) [?:?] Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: unknown_ca at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?] at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?] at sun.security.ssl.TransportContext.fatal(TransportContext.java:307) ~[?:?] at sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) ~[?:?] at sun.security.ssl.TransportContext.dispatch(TransportContext.java:180) ~[?:?] at sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) ~[?:?] at sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:681) ~[?:?] at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:636) ~[?:?] at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:454) ~[?:?] at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:433) ~[?:?] at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:634) ~[?:?] at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:295) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1301) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1203) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1247) ~[netty-handler-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.32.Final.jar:4.1.32.Final] ... 15 more Python code to connect to the elastic client ``` Below is the code I used to connect to ES from python. ``` self.ESClient = connections.create_connection( hosts='10.8.1.205', port=9650, http_auth=('elastic','testpassword'), connection_class=RequestsHttpConnection, scheme='https', use_ssl=True, verify_certs=True, ca_cert = '/etc/kibana/config/client-ca.cer', client_cert = '/etc/kibana/config/client.cer', client_key = '/etc/kibana/config/client.key', timeout=self.ES_TIMEOUT) ``` Any help on the above issue is much appreciated. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 29, 2019, 6:16am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/2 "2019-05-29T06:16:23Z")
Hi there @udara, TLS can be used both to - "Authenticate" the server in a connection. This is what usually happens when you navigate to a website with your browser and your browser validates that the certificate that the server on the other side presents is valid and signed by a trusted CA. - Authenticate the client in the connection. This is also called "mutual TLS authentication" and it means that the client also will use a certificate and key pair in order to authenticate itself to the server. Do you also need to do mutual TLS authentication, or are you interested in setting TLS for encrypting communications between the clients and Elasticsearch and you want the clients to authenticate themselves with username and password? I'm just trying to clarify because setting both ```auto elasticsearch.ssl.certificate: /etc/kibana/config/client.cer elasticsearch.ssl.key: /etc/kibana/config/client.key ``` and ```auto elasticsearch.username: "elastic" elasticsearch.password: "testpassword" ``` seems unnecessary. Now, the error that you see in your Python client, basically says that it doesn't trust the certificate that Elasticsearch is presenting for TLS. You have set ```auto ca_cert = '/etc/kibana/config/client-ca.cer', ``` but this ca\_cert file should be the CA cert that has signed the certificate that you have configured Elasticsearch with, and _not_ the CA cert that has signed the client certificates. Can you share your elasticsearch.yml too ? ---
**Author:** ![udara](https://avatars.discourse-cdn.com/v4/letter/u/65b543/32.png) [@udara](https://discuss.elastic.co/u/udara)\ **Post date:** [May 29, 2019, 9:27am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/3 "2019-05-29T09:27:43Z")
> [@ikakavas](#): > > Can you share your elasticsearch.yml too ? In elasticsearch.yml I have set up the following configurations. ``` xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 ``` The certificates were generated from the steps mentioned in the below page > **[Configure TLS | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates)** But to generate a client certificate we used the steps mentioned in the below link. > **[Elasticsearch Security: Configure TLS/SSL & PKI Authentication](https://www.elastic.co/blog/elasticsearch-security-configure-tls-ssl-pki-authentication)** > > Learn how to enable Elasticsearch security, configure TLS/SSL, use PKI for authentication, authenticate Kibana to an Elasticsearch cluster using PKI, and set passwords for built-in users. under Creating a client certificate. So I believe its the same CA that signs both certificates that are configured in ES as well as the client certs. And I think that is why kibana is working with Elasticsearch. Because both are signed with the CA elastic-stack-ca.p12. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 29, 2019, 9:56am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/4 "2019-05-29T09:56:52Z")
> [@udara](#): > > In elasticsearch.yml I have set up the following configurations. You have enabled TLS only for the transport layer and _not_ for the http layer in Elastisearch. You are missing : [Configure TLS | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#tls-http) or you didn't share that part with us. > [@udara](#): > > And I think that is why kibana is working with Elasticsearch It looks like Kibana works because it connects over http, and not https. What is the value for ```auto elasticsearch.hosts: ``` in kibana.yml ? We can only guess if you don't share the whole configuration in elasticsearch.yml and kibana.yml 🙂 > [@udara](#): > > So I believe its the same CA that signs both certificates that are configured in ES as well as the client certs. Maybe I read the above error wrong and ```auto Received fatal alert: unknown_ca ``` is Elasticsearch complaining that it can't validate the certificate that the python client is sending with `client_cert` which would be possible depending on the configuration you have for TLS on the http layer of ES. Please share all applicable parts from elasticsearch.yml and kibana.yml and do specify if you _want to_ use mutual TLS authentication for your clients connecting to Elasticsearch and we'll get to the bottom of this. ---
**Author:** ![udara](https://avatars.discourse-cdn.com/v4/letter/u/65b543/32.png) [@udara](https://discuss.elastic.co/u/udara)\ **Post date:** [May 29, 2019, 10:21am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/5 "2019-05-29T10:21:15Z")
> [@ikakavas](#): > > which would be possible depending on the configuration you have for TLS on the http layer of ES. > > Please share all applicable parts from elasticsearch.yml and kibana.yml and do specify if you _want to_ use mutual TLS authentication for your clients connecting to Elasticsearch and we'll get to the bottom of this. Hi @ikakavas My bad for not sharing the whole ymls. But I'm using TLS in the http as well as the transport layer as well. the full elasticsearch.yml ``` node.name: node-1 cluster.initial_master_nodes: node-1 path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 0.0.0.0 http.port: 9650 xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12 xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12 ``` Complete kibana.yml ``` elasticsearch.hosts: "https://10.8.1.205:9650" xpack.security.enabled: true elasticsearch.username: "elastic" elasticsearch.password: "testpassword" elasticsearch.ssl.certificate: /etc/kibana/config/client.cer elasticsearch.ssl.key: /etc/kibana/config/client.key server.ssl.enabled: true server.ssl.certificate: /etc/kibana/config/kibana-server/kibana-server.crt server.ssl.key: /etc/kibana/config/kibana-server/kibana-server.key server.host: 10.8.1.205 server.port: 8798 ``` And both kibana \> elasticsearch and browser \> kibana are working on https. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 30, 2019, 7:01am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/6 "2019-05-30T07:01:40Z")
Thanks for the info @udara. Would you also please specify if you _want_ to use TLS client authentication ? So that we don't try to solve a problem you do not have after all 🙂 See my original post for a description of what TLS client (or mutual) authentication is. If you do want to use TLS Client authentication, there are parts in your configuration missing, you also need to set up a [PKI Realm](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-pki-realm.html). If you don't want to use TLS client authentication, then you have extra configuration that you don't need , i.e. ```auto elasticsearch.ssl.certificate: /etc/kibana/config/client.cer elasticsearch.ssl.key: /etc/kibana/config/client.key ``` in kibana.yml But focusing in the problem at hand (unknown\_ca) , can you share the output of ```auto openssl s_client -showcerts -host 10.8.1.205 -port 9650 ``` and ```auto curl -v 'https://10.8.1.205:9650' --cacert client-ca.cer ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 27, 2019, 7:03am UTC](https://discuss.elastic.co/t/ssl-certificate-error-when-connecting-to-elastic-search-from-python/182977/7 "2019-06-27T07:03:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.