Watcher showing security_exception error

Elastic Stack Elasticsearch
1

Hello,
I have been facing an issue with some watchers regarding role privileges due to which the watchers don't trigger and show error. But I have superuser privileges.

"result": {
    "execution_time": "2024-06-13T07:39:04.586Z",
    "execution_duration": 4,
    "input": {
      "type": "search",
      "status": "failure",
      "error": {
        "root_cause": [
          {
            "type": "security_exception",
            "reason": "action [indices:data/read/search] is unauthorized for user [ssogd@company_name.com] with effective roles [superuser], this action is granted by the index privileges [read,all]"
          }
        ],
        "type": "security_exception",
        "reason": "action [indices:data/read/search] is unauthorized for user [ssogd@company_name.com] with effective roles [superuser], this action is granted by the index privileges [read,all]",
        "caused_by": {
          "type": "illegal_argument_exception",
          "reason": "node [special-2] does not have the remote cluster client role enabled"
        }
      },

Also, please note that special-2 is a local elasticsearch node and not a remote node.
Kindly help me with this. Thanks.

2

Are you trying to search on a remote cluster?

3

Hi Tim,

Yes I am searching on various remote clusters.

4

node [special-2] does not have the remote cluster client role enabled

The node you are using (special-2) does not have the remote_cluster_client role. That means it is not allowed to make connections to other clusters, therefore you cannot perform CCS queries from this node.

Please read the documentation for remote_cluster_client here:

5

Thanks for the prompt response, Tim.

We have recently upgraded all the clusters from 7.17 to 8.13 and the watcher was not giving any error before upgrade. May I please know what changed post upgrade. Please note that pre and post upgrade, we did not make any changes in elasticsearch.yml.

Also, if I create the same watcher again, will it execute on some other node, instead of special-2?