# Filebeat doesn't work with AWS ES domain

**URL:** https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683
**Category:** Beats
**Tags:** filebeat
**Created:** [November 17, 2020, 11:52am UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683 "2020-11-17T11:52:28Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![hairongGao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haironggao/32/79158_2.png) [@hairongGao](https://discuss.elastic.co/u/hairongGao)
#### Post date: [November 17, 2020, 11:52am UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683/1 "2020-11-17T11:52:28Z")

</div>

Hi Team,  
I've been trying to make filebeat working with AWS ES domain , but I keep getting this error:

> 2020-11-17T11:42:25.740Z ERROR [publisher\_pipeline\_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch([https://search-pogo-logging-3oegsu4hacueitzstmqkmj2jkm.us-east-1.es.amazonaws.com:443](https://search-pogo-logging-3oegsu4hacueitzstmqkmj2jkm.us-east-1.es.amazonaws.com:443))): Connection marked as failed because the onConnect callback failed: cannot retrieve the elasticsearch license from the /\_license endpoint, Filebeat requires the default distribution of Elasticsearch. Please make the endpoint accessible to Filebeat so it can verify the license.: unauthorized access, could not connect to the xpack endpoint, verify your credentials

Here is the configurations for filebeat:

```auto
 ---
apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: container
      paths:
        - /var/log/containers/*.log
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

    # To enable hints based autodiscover, remove `filebeat.inputs` configuration and uncomment this:
    #filebeat.autodiscover:
    # providers:
    # - type: kubernetes
    # node: ${NODE_NAME}
    # hints.enabled: true
    # hints.default_config:
    # type: container
    # paths:
    # - /var/log/containers/*${data.kubernetes.container.id}.log

    processors:
      - add_cloud_metadata:
      - add_host_metadata:

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}
    setup.ilm.enabled: false

```

I'm using [docker.elastic.co/beats/filebeat:7.10](http://docker.elastic.co/beats/filebeat:7.10) and AWS ES 7.8 , any comments/helps will be really appreciated!

---

<div class="post-metadata">

### Author: ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)
#### Post date: [November 17, 2020, 1:49pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683/2 "2020-11-17T13:49:06Z")

</div>

Hi,

Welcome to this forum! 🥳

I think you have the wrong FileBeat version: ElasticSearch on AWS is not the same as the Elastic ElasticSearch. Both build upon the opensource code of ElasticSearch but they are not the same. Because of this the closedsource version of Elastic FileBeat is incompatible with the AWS Elasticsearch - in your case it fails to read the license endpoint which does not exist.

Therefore, you cannot use the standard version of FileBeat - you have to use the opensource version(I did not find the 7.10 version yet): [docker.elastic.co/beats/filebeat-oss:7.9.3](http://docker.elastic.co/beats/filebeat-oss:7.9.3)

Best regards  
Wolfram

---

<div class="post-metadata">

### Author: ![hairongGao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/haironggao/32/79158_2.png) [@hairongGao](https://discuss.elastic.co/u/hairongGao)
#### Post date: [November 17, 2020, 3:37pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683/3 "2020-11-17T15:37:26Z")

</div>

YOU.ARE.THE.MAN 👍  
Thanks so much for this, I've been trapped for 3 hrs and finally it's working!

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 17, 2020, 8:12pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683/4 "2020-11-17T20:12:09Z")

</div>

> [@Wolfram\_Haussig](#):
>
> Because of this the closedsource version of Elastic FileBeat

FYI there is no closed source version of Filebeat.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 15, 2020, 10:12pm UTC](https://discuss.elastic.co/t/filebeat-doesnt-work-with-aws-es-domain/255683/5 "2020-12-15T22:12:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
