Cannot establish secure (SSL/TLS) connection between elasticsearch clusters

[iShiBin (Bin)]

Hi, I followed the guide on Elastic website "Adding the CA certificate from the local cluster as a trusted CA in each remote cluster (see Transport TLS settings)."

However, I found the connection could not be made. Maybe it is because both of my local and remote clusters are using SSL. After I applied the CA to each coordinator and gateway nodes, they can be connected.

Is this the right way to do this? If positive, will you please update your documentation as well?

[DavidTurner]

Hi @iShiBin,

First up, the docs you linked are for version 6.2. The latest version of this information is in the Elastic Stack docs here. However I don't see any major differences in the section you're talking about.

It's certainly possible to set up cross-cluster search with TLS enabled everywhere. Can you explain a bit more precisely where you got stuck? When you say "the CA" what do you mean? Do you have one CA shared by both clusters, or does each cluster have its own CA? What do you mean by "applied"? Can you share the commands you used to do this?

I tried following the docs myself and I did get stuck in two places:

• the trust relationship actually needs to be bidirectional. Each cluster needs to trust the other cluster's CA.

• I did not know how to add a CA certificate as a trusted CA in another cluster.

I think we can improve the docs in this area.

[iShiBin (Bin)]

Here are my answers to your questions:

Can you explain a bit more precisely where you got stuck?

After I import the local CA file (ca.pem) in the remote cluster, I cannot send query the data from the coordinator node in the local cluster. Then when I tried to restart this coordinator, I got Java exceptions attached as below.

Caused by: java.lang.IllegalStateException: failed to connect to remote clusters
at org.elasticsearch.transport.RemoteClusterService.initializeRemoteClusters(RemoteClusterService.java:340) ~[elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.transport.TransportService.doStart(TransportService.java:231) ~[elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:66) ~[elasticsearch-6.2.4.jar:6.2.4]

When you say "the CA" what do you mean?

I mean the CA PEM file.

Do you have one CA shared by both clusters, or does each cluster have its own CA?

Each cluster has its own CA file. And I don't think the CA needs to be imported/applied if they use the same one.

What do you mean by "applied"?

I mean import to the key store.

In addition, I think you get my point here: the trust relationship actually needs to be bidirectional. Each cluster needs to trust the other cluster's CA (if they don't use/share the same CA.)