# Shield 2.3.5 specifcally transport SSL/TLS failing
**URL:** https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728
**Category:** Elasticsearch
**Tags:** elastic-stack-security
**Created:** [August 10, 2016, 4:53pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728 "2016-08-10T16:53:25Z")
**Posts on this page:** 18
**Page:** 1
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 10, 2016, 4:53pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/1 "2016-08-10T16:53:25Z")
Trouble figuring out exactly what is going on but here is what I am seeing:
- shield http TLS/SSL working fine
- ES 2.3.5
- shield 2.3.5
- private CA
- shield transport failing with a message that typically is associated with uncommon CA
- unable to find valid certification path to requested target
- javax.net.ssl.SSLException: Received fatal alert: certificate\_unknown
- confirmed root CA is the same via SHA signature in keystores and md5sum
- created and configured a truststore specifically to troubleshoot , no change
- same keystore as the http is using
- set shield.transport.ssl: false , everything works
- this is a tribe node to another cluster
- curl to 9300 did end in an curl (58) nss error, while to 9200 works fine
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 10, 2016, 5:34pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/2 "2016-08-10T17:34:05Z")
Is SSL only failing on your tribe node?
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 10, 2016, 5:53pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/3 "2016-08-10T17:53:41Z")
no , all nodes for transport.ssl
between tribe and other single node clusters
The tribe node also has a second ES instance for kibana they share the JKS and it isn't working there either
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 10, 2016, 8:16pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/4 "2016-08-10T20:16:47Z")
Can you provide your full shield config in the elasticsearch.yml and the output of `keytool -list -v -keystore yourkeystore.jks` for two of the nodes? Maybe we can just try to get the two instances on the tribe node to communicate.
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 10, 2016, 11:21pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/5 "2016-08-10T23:21:41Z")
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 3 entries
Alias name: tribe-cert
Creation date: Jul 27, 2016
Entry type: trustedCertEntry
Owner: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=ST
Issuer: C=US, O=company LLC, OU=companyIT, CN=company Server CA
Serial number: 5a8583f1c81654b6
Valid from: Wed Jul 27 08:21:58 MST 2016 until: Fri Jul 27 08:21:58 MST 2018
Certificate fingerprints:
MD5: E2:9B:26:00:ED:FD:66:DD:1C:6E:D5:BA:2F:08:ED:71
SHA1: 76:4E:5E:63:FB:88:93:99:EF:71:48:63:EE:AB:CC:0E:62:20:B7:D2
SHA256: 2C:A0:4C:E5:F4:E6:D4:8A:ED:68:4F:0A:7B:7D:8C:91:D8:B4:1E:43:71:0A:CF:AF:51:CD:38:21:AC:0C:F9:58
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
redacted ...\<
]
]
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:false
PathLen: undefined
]
#3: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
[DistributionPoint:
[URIName: [http://redacted](http://redacted)]
CRLIssuer:[C=US, O=company LLC, OU=companyIT, CN=company Server CA]
]]
#4: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
clientAuth
1.3.6.1.5.5.7.3.21
0.4.0.2231.3.0
]
#5: ObjectId: 2.5.29.46 Criticality=false
FreshestCRL [
[DistributionPoint:
[URIName: [http://redacted](http://redacted)
]]
#6: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Non\_repudiation
Key\_Encipherment
Data\_Encipherment
]
#7: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: [elk-tribe.company.us](http://elk-tribe.company.us)
IPAddress: 192.168.255.161
]
#8: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
redacted l...
]
]
* * *
* * *
Alias name: tribe-key
Creation date: Jul 27, 2016
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=ST
Issuer: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=ST
Serial number: 26b233c6
Valid from: Wed Jul 27 08:23:32 MST 2016 until: Mon Jul 09 08:23:32 MST 2018
Certificate fingerprints:
MD5: B8:B9:A1:8E:C1:E0:9C:21:3D:FF:BF:6F:54:88:1C:2B
SHA1: A6:C6:28:FD:AD:9D:5F:BF:E6:8F:13:34:B8:7A:F8:58:C2:26:89:D7
SHA256: 8C:F3:3E:98:F0:94:FE:F9:01:BC:AF:49:C3:6D:DB:D2:5A:34:DF:AF:68:C0:46:69:32:81:23:72:36:83:80:70
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
Redated
]
]
* * *
* * *
Alias name: company\_ca
Creation date: Jul 27, 2016
Entry type: trustedCertEntry
Owner: C=US, O=company LLC, OU=companyIT, CN=company Server CA
Issuer: C=US, O=company LLC, OU=companyIT, CN=company Server CA
Serial number: 4694d204c830a425
Valid from: Wed Dec 17 13:36:26 MST 2014 until: Sat Dec 17 13:36:26 MST 2044
Certificate fingerprints:
MD5: F0:CB:31:05:0E:05:86:3C:FA:E3:9A:61:CE:F1:B2:9F
SHA1: 2E:92:93:A7:10:AB:00:FB:1B:DF:BC:2C:FF:AA:89:CF:B6:DD:CF:68
SHA256: CA:8F:51:1D:0C:F1:CD:71:93:2A:13:D1:DD:FE:1A:CC:42:6E:F0:C3:1C:94:8E:2D:21:ED:1A:E0:09:F7:A5:26
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
redacted
]
]
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Key\_CertSign
Crl\_Sign
]
#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
redacted
]
* * *
* * *
shield.audit.enabled: true
shield.ssl.keystore.path: "/etc/elasticsearch/elktribe.jks"
# shield.ssl.truststore.path: "/etc/elasticsearch/elktrust.jks"
# shield.ssl.truststore.password: "redacted"
shield.ssl.keystore.password: "redacted"
shield.ssl.keystore.key\_password: "redacted"
shield.transport.ssl: false
shield.http.ssl: true
shield.ssl.ciphers: TLS\_RSA\_WITH\_AES\_128\_CBC\_SHA256,TLS\_RSA\_WITH\_AES\_128\_CBC\_SHA
# shield.ssl.hostname\_verification.resolve\_name: false
shield. authc:
realms:
file1:
type: file
order: 0
enable: true
files:
users: /etc/elasticsearch/shield/users
users\_roles: /etc/elasticsearch/shield/users\_roles
# ldap1:
# type: ldap
# order: 1
# enabled: false
# url: 'url\_to\_ldap1'
# ldap2:
# type: ldap
# order: 2
# enabled: false
# url: 'url\_to\_ldap2'
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 11, 2016, 12:00am UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/6 "2016-08-11T00:00:14Z")
From the tribe config from the yml
kibana:
[cluster.name](http://cluster.name): company-kibana
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9309"]
discovery.zen.ping.multicast.enabled: false
shield.ssl.keystore.path: "/etc/elasticsearch/elktribe.jks"
# shield.ssl.truststore.path: "/etc/elasticsearch/elktrust.jks"
# shield.ssl.truststore.password: "redacted"
```
shield.ssl.keystore.password: "redacted"
shield.ssl.keystore.key_password: "redacted"
shield.transport.ssl: false
shield.http.ssl: true
shield.ssl.ciphers: TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA
```
# shield.ssl.hostname\_verification.resolve\_name: false
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 11, 2016, 12:13pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/7 "2016-08-11T12:13:45Z")
> [@datadude](#):
>
> Alias name: tribe-keyCreation date: Jul 27, 2016Entry type: PrivateKeyEntryCertificate chain length: 1Certificate[1]:Owner: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=STIssuer: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=ST
It looks like you are using a self-signed certificate and not one signed by the CA. Is this what you intended to use?
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 11, 2016, 12:49pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/8 "2016-08-11T12:49:38Z")
It isn't a self signed certificate but a private CA is used. The output is
sanitized a bit. If you notice the issuer of the tribe-cert is the company
server ca, the car cert is the last cert of the output "Alias name:
company\_ca"
> jaymode Jay Modi
> August 11
> datadude:
>
> > Alias name: tribe-keyCreation date: Jul 27, 2016Entry type:
> > PrivateKeyEntryCertificate chain length: 1Certificate[1]:Owner: CN=
> > [elk-tribe.company.us](http://elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=STIssuer: CN=
> > [elk-tribe.company.us](http://elk-tribe.company.us), OU=IT, O=company, L=City, ST=State, C=ST
>
> It looks like you are using a self-signed certificate and not one signed
> by the CA. Is this what you intended to use?
>
> * * *
>
> Visit Topic or reply to this email to respond.
>
> * * *
>
> In Reply To
> datadude
> August 10
> Keystore type: JKS Keystore provider: SUN Your keystore contains 3
> entries Alias name: tribe-cert Creation date: Jul 27, 2016 Entry type:
> trustedCertEntry Owner: [CN=elk-tribe.company.us](http://CN=elk-tribe.company.us), OU=IT, O=company, L=City,
> ST=State, C=ST Issuer: C=US, O=company LLC, OU=companyIT, CN=company Server
> CA Se…
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 11, 2016, 1:20pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/9 "2016-08-11T13:20:39Z")
Yes, the issuer of `tribe-cert` entry is not self signed, but the certificate for the `privateKeyEntry` is self-signed, unless that was part of the sanitization. If I had to guess, I think the tribe-cert is actually what should be associated with the `privateKeyEntry`, which represents both the private key and certificate that will be used by the node. If that was the intent, you can import the signed certificate with the alias `tribe-key`.
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 11, 2016, 1:42pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/10 "2016-08-11T13:42:03Z")
OK yes a different alias was used at import, I will correct this and try
again.
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 12, 2016, 11:54pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/11 "2016-08-12T23:54:11Z")
seemingly fixed the certificate misstep but it still doesn't seem the communication with transport.ssl enabled is working. I am just getting timeout errors in elasticsearch...
Tribe log
[2016-08-12 16:42:47,162][DEBUG][action.admin.indices.create] [tribe1] no known master node, scheduling a retry
[2016-08-12 16:43:17,163][DEBUG][action.admin.indices.create] [tribe1] timed out while retrying [indices:admin/create] after failure (timeout [30s])
[2016-08-12 16:43:17,163][WARN][rest.suppressed] path: /.kibana, params: {index=.kibana}
MasterNotDiscoveredException[null]
kibana es instance log
2016-08-12 16:43:19,422][WARN][discovery.zen] [kibana-only] failed to validate incoming join request from node [{tribe1/kibana}{GxyIe0YeSRKhJWWHfB2OuA}{10.136.255.161}{10.136.255.161:9301}{data=false, client=true}]
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 15, 2016, 10:51am UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/12 "2016-08-15T10:51:02Z")
In the tribe log, do you see any exceptions or errors related to the joining to the kibana cluster? Does the tribe client have the same cluster name as the kibana cluster?
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 15, 2016, 3:26pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/13 "2016-08-15T15:26:50Z")
There is a "failed to send join request", see below
The tribe client is not the same as the cluster name, should it be?
The tribe client name is simply "kibana" and the kibana cluster name is "kibana-only"
[2016-08-15 08:03:13,999][DEBUG][action.admin.indices.create] [tribe1] no known master node, scheduling a retry
**[2016-08-15 08:03:31,849][INFO][discovery.zen] [tribe1/kibana] failed to send join request to master [{kibana-only}{AjBZwaHRSNaBxPYrilWxHw}{127.0.0.1}{127.0.0.1:9309}], reason [ElasticsearchTimeoutException[Timeout waiting for ta**
**sk.]]**
[2016-08-15 08:03:43,999][DEBUG][action.admin.indices.create] [tribe1] timed out while retrying [indices:admin/create] after failure (timeout [30s])
[2016-08-15 08:03:43,999][WARN][rest.suppressed] path: /.kibana, params: {index=.kibana}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:226)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[2016-08-15 08:03:51,558][DEBUG][action.admin.indices.create] [tribe1] no known master node, scheduling a retry
[2016-08-15 08:04:21,558][DEBUG][action.admin.indices.create] [tribe1] timed out while retrying [indices:admin/create] after failure (timeout [30s])
[2016-08-15 08:04:21,558][WARN][rest.suppressed] path: /.kibana, params: {index=.kibana}
MasterNotDiscoveredException[null]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:226)
at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:236)
at org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:804)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 15, 2016, 5:34pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/14 "2016-08-15T17:34:17Z")
What I mean is if you specify the correct the cluster name in your tribe settings. So in your case it would be something like:
```
tribe:
kibana:
cluster.name: 'kibana-only'
```
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 15, 2016, 6:06pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/15 "2016-08-15T18:06:08Z")
I do have it the way you give in your example.
I was errant in the last post the "kibana-only" is the node name.
Everything still seems to work fine with the transport.ssl: false, it is only when enable is there an issue.
The tribe config is still the same as pasted above in post 6
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [August 15, 2016, 6:19pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/16 "2016-08-15T18:19:09Z")
Can you stop both instances, and then capture the logs from when you start both back up and share the logs as a gist?
---
### Author:  [@datadude](https://discuss.elastic.co/u/datadude)
#### Post date: [August 15, 2016, 8:16pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/17 "2016-08-15T20:16:26Z")
found it thanks for your feedback.
It was another SAN error, easier to fix it with a hostname change to match FQDN.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:42pm UTC](https://discuss.elastic.co/t/shield-2-3-5-specifcally-transport-ssl-tls-failing/57728/18 "2017-07-06T13:42:17Z")