Elastic-search SSL certification error, unable to open kibana

Elastic Stack Elasticsearch
1

I'm unable to open kibana as elastic-search is irresponsive as I made changes to elasticsearch.yml for SSL authentication.
On dashboard on webpage i get "Cannot connect to the Elasticsearch cluster".

My elasticsearch.yml looks like

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.enabled: true
xpack.ssl.verification_mode: none
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.key
xpack.security.http.ssl.certificate: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.crt
xpack.security.http.ssl.certificate_authorities: [ "/usr/share/ca-certificates/mozilla/DigiCert_Trusted_Root_G4.crt" ]
xpack.security.transport.ssl.key: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.key
xpack.security.transport.ssl.certificate: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.crt
xpack.security.transport.ssl.certificate_authorities: [ "/usr/share/ca-certificates/mozilla/DigiCert_Trusted_Root_G4.crt" ]

error i get is also

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
labuser@illumni8:/root/elasticsearch-7.1.1/config$ curl -v 'https://10.27.0.4:9200'

  • Expire in 0 ms for 6 (transfer 0x55e27d9cf5c0)
  • Trying 10.27.0.4...
  • TCP_NODELAY set
  • Expire in 200 ms for 4 (transfer 0x55e27d9cf5c0)
  • Connected to 10.27.0.4 (10.27.0.4) port 9200 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • successfully set certificate verify locations:
  • CAfile: none
    CApath: /etc/ssl/certs
  • TLSv1.3 (OUT), TLS handshake, Client hello (1):
  • TLSv1.3 (IN), TLS handshake, Server hello (2):
  • TLSv1.2 (IN), TLS handshake, Certificate (11):
  • TLSv1.2 (OUT), TLS alert, unknown CA (560):
  • SSL certificate problem: unable to get local issuer certificate
  • Closing connection 0
    curl: (60) SSL certificate problem: unable to get local issuer certificate
    More details here: curl - SSL CA Certificates

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

On elasticsearch logs , I get

[2019-08-07T23:02:29,687][WARN ][o.e.h.AbstractHttpServerTransport] [illumni8] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=0.0.0.0/0.0.0.0:9200, remoteAddress=/10.27.0.4:42440}

[2019-08-07T23:02:28,702][WARN ][o.e.h.AbstractHttpServerTransport] [illumni8] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=0.0.0.0/0.0.0.0:9200, remoteAddress=/127.0.0.1:41492}
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 48454144202f20485454502f312e310d0a417574686f72697a6174696f6e3a2042617369632064584e6c636a707759584e7a643239795a413d3d0d0a486f73743a203132372e302e302e313a393230300d0a436f6e74656e742d4c656e6774683a20300d0a436f6e6e656374696f6e3a206b6565702d616c6976650d0a0d0a

Thank-you.

2

Hi Asad,

I am following this thread from your last post "Unable to configure 'x-pack security' under basic license. I am working on a similar setup in Kubernetes elastic(3-node) v7.1.0 and kibana. I am in the early state of setup, going through the parameters for elasticsearch.yml. I don't see the license parameter in your file, are you still doing it with basic license.
xpack.license.self_generated.type: "basic"
Another thing, I am getting some weird errors while generating certs on k8s hosts. Did you face any issues there?

Thanks,
Harry

1 Like
3

You show us how you have configured Elasticsearch, but now if and how you have configured kibana to connect to Elasticsearch now that the http layer is configured for TLS.
Please see Setting up TLS on a cluster | Elasticsearch Guide [7.3] | Elastic and step 2 in Encrypting communications in Kibana | Kibana Guide [7.3] | Elastic

Can you share your kibana.yml ?

This is just a client ( could be Kibana or a browser request ) attempting to connect to Elasticsearch over http when Elasticsearch is already configured to expect connections with http over TLS

1 Like
4

Thanks I finally solved with this .yml on both sides. but I'm not password prompt , its is said its not available under "basic license"?

xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.enabled: false
 #xpack.ssl.verification_mode: none
#xpack.security.transport.ssl.verification_mode: certificate
#xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: cert/key.pem
xpack.security.http.ssl.certificate: cert/star_inboxbiz_com.pem

and kibanna.yml

server.ssl.enabled: true
server.ssl.key: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.key
server.ssl.certificate: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.crt
5

Looks like you didn't solve that problem, instead you just disabled security in Elasticsearch with

xpack.security.enabled: false

The changes you made to Kibana also, have nothing to do with the communication between Kibana and Elasticsearch but with the communication between your browser and Kibana.

I would urge you to go through and read the available documetation ( Setting up TLS on a cluster | Elasticsearch Guide [7.3] | Elastic) in detail before moving forward with the rest of the configuration. It is a small upfront time investment that will pay off significantly when you actually try and configure this when knowing what needs to be configured and how ! We'll be happy to help with specific issues then :slight_smile:

1 Like
6

For 1 week whole spend long nights troubleshooting , it nothing seems to work. My mistake on

xpack.security.enabled: false

i change back the settings, now i get same error as before.

8

I'm not sure if this helps, but we had some problems around ES+Kibana setup after enabling authentication.

At first we enabled an anonymous user on ES, with superuser role, and everything went fine. Then we changed the role of the anonymous user and Kibana started to have problems. It was because we didn't add the ES username+password to the kibana config file and that's needed from Kibana to perform some operations on startup.

AFter that, everything works fine.

edit: ahhh, your problems seems to be around connection issues and not auth issues. Then probably my answer is not helpful at all at the moment

1 Like
9

On further looking at logs
[2019-08-09T10:00:22,692][WARN ][o.e.h.AbstractHttpServerTransport] [node-1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=0.0.0.0/0.0.0.0:9200, remoteAddress=/172.27.0.5:65296}

This remote address is uknown to me, where is remoteaddress info picked on?

10

Thanks,

my recent error are host-name related.

network.host: 10.27.0.4
network.publish_host: 10.27.0.4
#http.publish_host: illumin8
#http.host: illumin8
transport.publish_port: 9200

xpack.security.enabled: true
#xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
#xpack.security.enabled: false
#xpack.ssl.verification_mode: none
#xpack.security.http.ssl.verification_mode: certificate

#xpack.security.http.ssl.key: cert/wildcard_inboxbiz_com.key
#xpack.security.http.ssl.certificate: cert/wildcard_inboxbiz_com.crt
#xpack.security.http.ssl.certificate_authorities: [ "cert/DigiCertCA2.pem" ]
#xpack.security.http.ssl.certificate_authorities: cert/DigiCertCA.crt

xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: cert/wildcard_inboxbiz_com.key
xpack.security.transport.ssl.certificate: cert/wildcard_inboxbiz_com.crt
xpack.security.transport.ssl.certificate_authorities: [ "cert/DigiCertCA2.pem" ]

xpack.security.transport.ssl.certificate_authorities: cert/DigiCertCA.crt

elasticsearch.hosts: ["https://10.27.0.4:9200"]

elasticsearch.username: "elastic"
elasticsearch.password: "elk#123"
elasticsearch.ssl.certificateAuthorities: [ "/root/elasticsearch-7.1.1/config/cert/DigiCertCA.crt" ]

Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.

These settings enable SSL for outgoing requests from the Kibana server to the browser.

server.ssl.enabled: true
server.ssl.key: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.key
server.ssl.certificate: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.crt
#lasticsearch.ssl.certificateAuthorities: [ "/root/elasticsearch-7.1.1/config/cert/DigiCertCA2.pem" ]

server.ssl.keyPassphrase: 12345

Optional settings that provide the paths to the PEM-format SSL certificate and key files.

These files validate that your Elasticsearch backend uses the same key files.

elasticsearch.ssl.certificate: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.crt
elasticsearch.ssl.key: /root/elasticsearch-7.1.1/config/cert/wildcard_inboxbiz_com.key

Optional setting that enables you to specify a path to the PEM file for the certificate

authority for your Elasticsearch instance.

elasticsearch.ssl.certificateAuthorities: [ "/root/elasticsearch-7.1.1/config/cert/DigiCertCA2.pem" ]

To disregard the validity of SSL certificates, change this setting's value to 'none'.

elasticsearch.ssl.verificationMode: certificate

11

Problem solved
with
elasticsearch.hosts: ["http://10.27.0.4:9200"]

was https, i got confused as most examples https://www.elastic.co/guide/en/kibana/current/configuring-tls.html
it mentioned https which was misleading since i m using all in one ELK is one single host.

Also, I'm not aware of any config on elasticsearch side to have it receive traffic as "https"?

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.