# "io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record" after ssl enabling my cluster

**URL:** <https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987>\
**Category:** Elasticsearch\
**Created:** [March 9, 2017, 1:20pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987 "2017-03-09T13:20:39Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guy\_Shilo](https://avatars.discourse-cdn.com/v4/letter/g/c0e974/32.png) [@Guy\_Shilo](https://discuss.elastic.co/u/Guy_Shilo)\
**Post date:** [March 9, 2017, 1:20pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/1 "2017-03-09T13:20:39Z")

</div>

Hello

I'm testing X-Pack security and I wanted to encrypt communication in my cluster (non-production).  
So I followed the instructions [here](https://www.elastic.co/guide/en/x-pack/current/ssl-tls.html#installing-node-certificates) and created self signed certificate for each node. I also changed/added all needed parameters in elasticsearch configuration file on each node.

Now, when I restart the cluster the nodes don't seem to talk to each other and I find this kind of errors in the log: "io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record".  
I googled it and understood that a client is trying to access a SSL enabled node via non-ssl connection. I double checked all nodes and they all configured to use ssl.

I even shut down all nodes except one, so there should'nt be any incoming traffic, but still I see the same error again and again in its log !!! What can be the reason ? what am I doing wrong ?

(If it matters, I am using a trial license, which I think is good for 30 days)

Thanks

Guy

---

<div class="post-metadata">

**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\
**Post date:** [March 14, 2017, 3:44pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/2 "2017-03-14T15:44:57Z")

</div>

Hello,

When opening an issue like this, it is important to provide log output, configuration, and versions of the product. Once that has been provided, we will be able to assist you with this.

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 5:31pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/3 "2017-03-21T17:31:13Z")

</div>

Same issue for me

The keys were generated from the sec\_course.zip  
all paths, names and ip addresses to setup the keys are correct and Ive done the exercise twice from the beginning to be sure  
...  
CODE

```
xpack:
    ssl:
        key: /usr/share/elasticsearch/bin/config/x-pack/nn775/nn775.key
        certificate: /usr/share/elasticsearch/bin/config/x-pack/nn775/nn775.crt
        # This points to where the CA cert is - every node shares this

```

certificate\_authorities: /usr/share/elasticsearch/bin/config/ca/ca.crt

```
# We need to ensure that the transport (port 9300) and HTTP (port
# 9200) protocols secure communication is enabled:
xpack:
    security:
        transport.ssl.enabled: true
        http.ssl.enabled: true

```

...  
CODE  
...

log is here  
bottom of page

[https://sites.google.com/site/developtroubleshooting/home/logs](https://sites.google.com/site/developtroubleshooting/home/logs)

[https://sites.google.com/site/developtroubleshooting/home/logs/not%20%20a%20tls%20record.txt?attredirects=0&d=1](https://sites.google.com/site/developtroubleshooting/home/logs/not%20%20a%20tls%20record.txt?attredirects=0&d=1)

---

<div class="post-metadata">

**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\
**Post date:** [March 21, 2017, 6:03pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/4 "2017-03-21T18:03:56Z")

</div>

This is HTTP traffic `o.e.x.s.t.n.SecurityNetty4HttpServerTransport`. What other settings do you have? What else is trying to connect using HTTP?

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 6:19pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/5 "2017-03-21T18:19:06Z")

</div>

> [@Bob\_Metelsky](#):
>
> /usr/share/elasticsearch/bin/config/ca/ca.crt

Im setting this up - per the course

I'm guessing this will protect the 9200-9300 ports of elasticsearch.  
This would be from either this  
[https://nn775:9200/sales\_record/\_search?pretty](https://nn775:9200/sales_record/_search?pretty)

or kibana tryining to connect to elastic

this is my elsticsearch.yml file

...  
CODE  
....  
xpack:  
ssl:  
key: /usr/share/elasticsearch/bin/config/x-pack/nn775/nn775.key  
certificate: /usr/share/elasticsearch/bin/config/x-pack/nn775/nn775.crt  
# This points to where the CA cert is - every node shares this  
certificate\_authorities: /usr/share/elasticsearch/bin/config/ca/ca.crt

```
    # We need to ensure that the transport (port 9300) and HTTP (port
    # 9200) protocols secure communication is enabled:
    xpack:
        security:
            transport.ssl.enabled: true
            http.ssl.enabled: true

    # AUDIT ########################################################
    xpack.security.audit.enabled: true
    xpack.security.audit.outputs: [index, logfile]
    xpack.security.audit.index.settings:
      index:
        number_of_shards: 1
        number_of_replicas: 1

    xpack.security.audit.logfile.events.exclude: [access_granted]
    xpack.security.audit.index.events.exclude: [access_granted]

    ###################################################################
    #xpack.monitoring.enabled: false
    #Set to false to disable X-Pack monitoring. Configure in both elasticsearch.yml and kibana.yml.

    #xpack.graph.enabled: false
    #Set to false to disable X-Pack graph. Configure in both elasticsearch.yml and kibana.yml.

    #xpack.watcher.enabled: false
    #Set to false to disable Watcher. Configure in elasticsearch.yml only.

    #xpack.reporting.enabled: false
    #Set to false to disable X-Pack reporting. Configure in kibana.yml only.

    bootstrap.system_call_filter: false
    cluster.name: EScluster-rack2
    node.name: ${HOSTNAME}
    node.master: true
    node.data: true
    network.host: ${HOSTNAME}
    http.port: 9200
    discovery.zen.minimum_master_nodes: 3
    path.data: /mnt/data/elk/5.2.data
    path.logs: /var/log/elasticsearch
    #
    # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
    # available on the system and that the owner of the process is allowed to use this limit.
    #
    discovery.zen.ping.unicast.hosts: ["nn775","sn776","dn777","dn778","dn779"]
    #
    # node.max_local_storage_nodes: 1
    # Require explicit names when deleting indices:
    # action.destructive_requires_name: true

```

...  
CODE  
...

I wasnt getting these errors before I added the certs to the elasticsearc.xml

---

<div class="post-metadata">

**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\
**Post date:** [March 21, 2017, 7:00pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/6 "2017-03-21T19:00:02Z")

</div>

> [@Bob\_Metelsky](#):
>
> or kibana tryining to connect to elastic

Sounds like this could be the source. Did you change kibana to use https and trust the cert?

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 7:03pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/7 "2017-03-21T19:03:04Z")

</div>

This is from the security course- lab 4 getting up ttls

KIbana was the next step - let me set that and see...

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 7:22pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/8 "2017-03-21T19:22:48Z")

</div>

same error in teh es.log

Ive added below to the kibana.yml

```
elasticsearch.url: "https://192.168.1.180:9200"
elasticsearch.ssl.verify: false 

console.proxyConfig:
 - match:
    host: "*"
    port: "{9200..9202}"

   ssl:
    ca: "/usr/share/elasticsearch/bin/config/ca/ca.crt"

```

restarted both kibana and es -

when I try this  
[https://nn775:9200/sales\_record/\_search?pretty](https://nn775:9200/sales_record/_search?pretty)

I get the expected behavior of this site cant be trusted - procede anyway?  
I do and am prompted to login.

Ive reset the password properly for elastic user and its not accepting the credentials  
ES log is still churning out this error not a SSL/TLS record ![](https://us1.discourse-cdn.com/elastic/original/3X/f/a/fa320fbd9bae04437465ff7996a1bce781e6dd2a.jpg)

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 7:44pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/9 "2017-03-21T19:44:40Z")

</div>

I removed the security settings and everything is working.

Ill try again - do you have access to the security course? chapter 4  
I beleive I followed it to the letter. Up until this everything was very accurate in the course

---

<div class="post-metadata">

**Author:** ![Bob\_Metelsky](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bob_metelsky/32/11387_2.png) [@Bob\_Metelsky](https://discuss.elastic.co/u/Bob_Metelsky)\
**Post date:** [March 21, 2017, 8:52pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/10 "2017-03-21T20:52:35Z")

</div>

so I regenerated the certs and keys again and I have ES working ok with teh certs - no errors and it blocks the [https://nn775:9200](https://nn775:9200)

Im getting errors to this effect

```
[2017-03-21T16:45:33,922][INFO][o.e.x.m.e.Exporters] [nn775] skipping exporter [default_local] as it is not ready yet
[2017-03-21T16:45:43,926][INFO][o.e.x.m.e.Exporters] [nn775] skipping exporter [default_local] as it is not ready yet
[2017-03-21T16:45:53,929][INFO][o.e.x.m.e.Exporters] [nn775] skipping exporter [default_local] as it is not ready yet
[2017-03-21T16:46:03,933][INFO][o.e.x.m.e.Exporters] [nn775] skipping exporter [default_local] as it is not ready yet

```

Ive made this entry to my kibana.yml  
elasticsearch.url: "[https://192.168.1.180:9200](https://192.168.1.180:9200)"  
elasticsearch.ssl.verify: false

```
console.proxyConfig:
 - match:
    host: "*"
    port: "{9200..9202}"

   ssl:
    ca: "/usr/share/elasticsearch/bin/config/ca/ca.crt"

```

browser says website cant be reached...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2017, 8:53pm UTC](https://discuss.elastic.co/t/io-netty-handler-ssl-notsslrecordexception-not-an-ssl-tls-record-after-ssl-enabling-my-cluster/77987/11 "2017-04-18T20:53:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.