# Can't connect to remote cluster

**URL:** https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468
**Category:** Elasticsearch
**Tags:** ccr-cross-cluster-replication
**Created:** [July 24, 2022, 4:47am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468 "2022-07-24T04:47:39Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Baygon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baygon/32/91792_2.png) [@Baygon](https://discuss.elastic.co/u/Baygon)
#### Post date: [July 24, 2022, 4:47am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/1 "2022-07-24T04:47:39Z")

</div>

Node 1: ES 7.14.0 (remote\_cluster\_client role is added to yml file)  
Node 2: ES 7.13.2

I've added my remote cluster via API:

```auto
PUT /_cluster/settings
{
  "persistent" : {
    "cluster" : {
      "remote" : {
        "cluster_one" : {    
          "seeds" : [
            "xxxxxxxx:9300" 
          ]
        }
      }
    }
  }
}

```

When checking connection I get:

```auto
{
  "cluster_one" : {
    "connected" : false,
    "mode" : "sniff",
    "seeds" : [
      "xxxxxxx:9300"
    ],
    "num_nodes_connected" : 0,
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s",
    "skip_unavailable" : false
  }
}

```

I verified firewall and connectivity between the clusters and everything is fine. Any idea where I could see some more detailed logs to debug this?

In the second cluster I don't see any transport port line so I guess it uses the default 3000.  
I also see:

```auto
network.host: ["0.0.0.0", 127.0.0.1", "[::1]"]
network.bind_host: 0.0.0.0
network.publish_host: 0.0.0.0

```

Shall I bind to the ip of host of the first cluster too ?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 24, 2022, 10:56pm UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/2 "2022-07-24T22:56:29Z")

</div>

If you set `network.host` it overrides all those other network settings and will apply for both transport and http ports.

> [@Baygon](#):
>
> Any idea where I could see some more detailed logs to debug this?

What do the Elasticsearch logs show?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [July 24, 2022, 11:14pm UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/3 "2022-07-24T23:14:38Z")

</div>

> [@Baygon](#):
>
> Shall I bind to the ip of host of the first cluster too ?

And No... you are not trying to join the 2 clusters into 1 cluster...

---

<div class="post-metadata">

### Author: ![Baygon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baygon/32/91792_2.png) [@Baygon](https://discuss.elastic.co/u/Baygon)
#### Post date: [July 25, 2022, 12:22pm UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/4 "2022-07-25T12:22:21Z")

</div>

> [@Baygon](#):
>
> network.host: ["0.0.0.0", 127.0.0.1", "[::1]"]  
> network.bind\_host: 0.0.0.0  
> network.publish\_host: 0.0.0.0

Ok I commented out all that and only kept:  
`network.host: ["0.0.0.0"]`

From cluster 1 I try `curl -XGET xx.xx.xx.xx:9200`  
I get 200 and the cluster status message, no issue.

Then I try `curl -XGET xx.xx.xx.xx:9300`

> curl: (1) Received HTTP/0.9 when not allowed

Trying the same on the cluster 2 using `curl -XGET 127.0.0.1:9300` and I get the same response, so obviously no connectivity issue.

I've tried to `tail -f elasticsearch.log` but no entries at all whenever I either call via curl, or activate the remote cluster synch on cluster 1

`netatst -plnt` on cluster 2 returns, so it listen to the right transport port:  
`tcp6 0 0 :::9300 :::* LISTEN 1939346/java`

---

<div class="post-metadata">

### Author: ![Baygon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baygon/32/91792_2.png) [@Baygon](https://discuss.elastic.co/u/Baygon)
#### Post date: [July 26, 2022, 10:12am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/5 "2022-07-26T10:12:42Z")

</div>

> [@warkolm](#):
>
> What do the Elasticsearch logs show?

there is absolutely nothing in the logs. (/var/log/elasticsearch.log for the 7.13.2, /home/ubuntu/elasticsearch-7.14.0/logs/my-cluster-prod.log)  
I tried monitoring the logs at the same time I was sending the api call to register the remote cluster, or even try to change settings from kibana remote cluster section, nothing happens in the logs of both machines

Nothing in syslog either on both machines

---

<div class="post-metadata">

### Author: ![Baygon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baygon/32/91792_2.png) [@Baygon](https://discuss.elastic.co/u/Baygon)
#### Post date: [July 27, 2022, 10:26am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/6 "2022-07-27T10:26:20Z")

</div>

ok I finally found something in the logs:

[2022-07-27T10:19:51,798][WARN][o.e.t.TcpTransport] [elasticsearch-node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,3,1), [Netty4TcpChannel{localAddress=/xxxx:9300, remoteAddress=/xxxxx:58478, profile=default}], closing connection

Cluster 1 has TLS enabled but not cluster 2

---

<div class="post-metadata">

### Author: ![Baygon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baygon/32/91792_2.png) [@Baygon](https://discuss.elastic.co/u/Baygon)
#### Post date: [July 27, 2022, 11:34am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/7 "2022-07-27T11:34:46Z")

</div>

I've enabled ssl/tls on the other node but now I get:

> [master-1] failed to establish trust with server at [\<unknown host\>]; the server provided a certificate with subject name [CN=instance] and fingerprint [xxx]; the certificate does not have any subject alternative names; the certificate is issued by [CN=Elastic Certificate Tool Autogenerated CA]; the certificate is signed by (subject [CN=Elastic Certificate Tool Autogenerated CA] fingerprint [yyyyy]) which is self-issued; the [CN=Elastic Certificate Tool Autogenerated CA] certificate is not trusted in this ssl context ([xpack.security.transport.ssl]); this ssl context does trust a certificate with subject [CN=Elastic Certificate Tool Autogenerated CA] but the trusted certificate has fingerprint [zzzz]
> 
> sun.security.validator.ValidatorException: PKIX path validation failed:  
> java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

Could it be that I generated the certificates the wrong way?  
`/usr/share/elasticsearch/bin/elasticsearch-certutil ca --out /etc/elasticsearch/certs/elk-cluster-ca.p12 --days 3650`

I could regenerate the CA and cert, but what should I put in instances.yml and hosts file for each machine?  
Or I should I use the same certs in both machines?

I don't really understand how to do that:

> - Adding the CA certificate from the local cluster as a trusted CA in each remote cluster (see [Transport TLS settings](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/security-settings.html#transport-tls-ssl-settings)).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 24, 2022, 11:34am UTC](https://discuss.elastic.co/t/cant-connect-to-remote-cluster/310468/8 "2022-08-24T11:34:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
