Failed to connect AWS ES cluster

Hi

We try to connect to AWS ES through file beat, However, the following errors comes up

pipeline/output.go:100	Failed to connect to backoff(elasticsearch(https://xxx.us-east-1.es.amazonaws.com:443)): 
Connection marked as failed because the onConnect callback failed: 
cannot retrieve the elasticsearch license: unauthorized access, 
could not connect to the xpack endpoint, verify your credentials

and the config file as

  filebeat.yml: |-
    filebeat.config:
      inputs:
        enabled: true
        path: ${path.config}/inputs.d/*.yml
        reload.enabled: true
        reload.period: 60s
      modules:
        enabled: true
        path: ${path.config}/modules.d/*.yml
        reload.enabled: true
        reload.period: 60s

    output.elasticsearch:
      protocol: "https"
      hosts: ["xxxx.us-east-1.es.amazonaws.com:443"]
      username: "user_id"
      password: "password"
      index: "test-log--%{+yyyy.MM.dd}"
    setup.template.name: "test-log"
    setup.template.pattern: "test-log-*"
    setup.ilm.enabled: false

The filebeat docker image is docker.elastic.co/beats/filebeat:7.13.1

The ES version of Amazon Elasticsearch Service is 7.10

Could someone help us to find the invalid configuration here?

Thanks

Please see Breaking changes in 7.13 | Beats Platform Reference [7.13] | Elastic

1 Like

Hi @warkolm

Thank you for your response, we connect to AWS ES with filebeat 7.10.2. The incorrect filebeat version is root cause.

FYI the aws service is a fork and we cannot provide support for it.

If you want to upgrade to our Elasticsearch Service you will be able to run the latest version of everything.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.