Hi all,
Been going through the doc, the topics and github. Can't seem to find an answer tho this one.
I have 2 clusters running in 2 different regions and wanted to test CCR. Both of them are running on 7.11.1. I was able to setup the remote clusters on both of them and can even do cross cluster searching on them. However, when I try to configure the cross cluster replication, I get the following error in Kibana:
Also tried it through the API:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"this node does not have the remote_cluster_client role"}],"type":"illegal_argument_exception","reason":"this node does not have the remote_cluster_client role"},"status":400}
I checked the logs and even tried setting the logger to trace. Unfortunately, nothing relevant to this comes up.
If I try to query the nodes API on both clusters, this is what I get:
In the ON region:
root@logs-ing-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/remote_cluster_client:true" | jq '._nodes'
{
"total": 0,
"successful": 0,
"failed": 0
}
#=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
root@logs-ing-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/ingest:true" | jq '._nodes'
{
"total": 2,
"successful": 2,
"failed": 0
}
#=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
root@logs-ing-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/ingest:true" | jq -c '.nodes[] | {"name": .name, "roles":.roles}'
{"name":"logs-ing-02.on","roles":["ingest","ml","remote_cluster_client","transform"]}
{"name":"logs-ing-01.on","roles":["ingest","ml","remote_cluster_client","transform"]}
In the QC region:
root@logs-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/remote_cluster_client:true" | jq '._nodes'
{
"total": 0,
"successful": 0,
"failed": 0
}
#=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
root@logs-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/ingest:true" | jq '._nodes '
{
"total": 2,
"successful": 2,
"failed": 0
}
#=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=
root@logs-02 ~: curl -nks "https://127.0.0.1:9200/_nodes/ingest:true" | jq -c '.nodes[] | {"name": .name, "roles":.roles}'
{"name":"logs-ing-01","roles":["ingest","ml","remote_cluster_client","transform"]}
{"name":"logs-ing-02","roles":["ingest","ml","remote_cluster_client","transform"]}
Both clusters can talk to each other on the http and transport ports and they are using the same CA for their certs.
Has anybody encountered this behaviour before and/or can anyone point me to how I can debug this?
