Logstash failed to create template on target server

[amralieg (Amr K Ali)]

Hi, I have the following logstash configuration, and when I run it it gives me the error.
the input server is hosted on elastic cloud v7.9 and the output server is hosted on AWS elasticsearch v7.8, any clue how to solve this error?

[2020-11-21T15:07:45,444][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. 
{:message=>"Got response code '401' contacting Elasticsearch at URL 
'https://xxx.eu-west-1.es.amazonaws.com:443/_xpack'"

this is the logstash config file

input {
    elasticsearch {
        hosts => ["https://yyy.eu-west-1.aws.found.io:443"]
        user => "xxx"
        password => "xxx"
        index => "idx-1,idx-2"
        size => 10
        scroll => "10m"
        codec => "json"
        docinfo => true
    }
}

filter {
    mutate { remove_field => [ "@version", "@timestamp" ] }
}

output {
    elasticsearch {
        document_id => "%{[@metadata][_id]}"
        hosts => ["https://xxx.es.amazonaws.com:443"]
        user => "xxx"
        password => "xxx"
        index => "%{[@metadata][_index]}"
    }
}

[Badger]

@amralieg

"Got response code '401' contacting Elasticsearch at URL 
'https://xxx.eu-west-1.es.amazonaws.com:443/_xpack'"

A 401 error is 'Unauthorized'. You have supplied a user/password, and a 401 error suggests that they are valid, but do not have the permissions required.

This document describes the roles that should be granted to the user used in an elasticsearch output.

[amralieg (Amr K Ali)]

Thanks, I'll look into this and see if it works.

[amralieg (Amr K Ali)]

I am using the super user (elastic) with the following permission, it is assigned to the all_access role which has the following definition, I am not sure what other permission needed for this user to be able to overcome the error above? My worry is that logstash tries to install a template that requires a specific plugin to be installed, is that the case?

{
  "description": "Allow full access to all indices and all cluster APIs",
  "index_permissions": [
    {
      "index_patterns": [
        "*"
      ],
      "fls": [],
      "masked_fields": [],
      "allowed_actions": [
        "*"
      ]
    }
  ],
  "tenant_permissions": [
    {
      "tenant_patterns": [
        "*"
      ],
      "allowed_actions": [
        "kibana_all_write"
      ]
    }
  ],
  "cluster_permissions": [
    "*"
  ]
}