# Metricbeat AWS module not sending data

**URL:** https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200
**Category:** Beats
**Tags:** beats-module, metricbeat
**Created:** [January 5, 2021, 12:00pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200 "2021-01-05T12:00:03Z")
**Posts on this page:** 19
**Page:** 1

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 5, 2021, 12:00pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/1 "2021-01-05T12:00:03Z")

</div>

Good day,

I'm currently trying to collect AWS metrics with the Metricbeat AWS module. And although the status of metricbeat looks fine, elasticsearch doesn't seem to receive any data.

my metricbeat.yml:

```auto
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

metricbeat.modules:
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<access-key-id>'
  secret_access_key: '<secret-access-key>'

- module: aws
  period: 300s
  metricsets:
    - cloudwatch
  metrics:
    - namespace: AWS/EC2
      resource_type: ec2:instance
  credential_profile_name: <my-profile-name>
  access_key_id: '<access-key-id>'
  secret_access_key: '<secret-access-key>'

- module: aws
  period: 24h
  metricsets:
    - billing
  access_key_id: '<access-key-id>'
  secret_access_key: '<secret-access-key>'
  cost_explorer_config:
    group_by_dimension_keys:
      - "AZ"
      - "INSTANCE_TYPE"
      - "SERVICE"
    group_by_tag_keys:
      - "aws:createdBy"

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: elastic
  password: <my-elastic-pw>
  ssl.certificate_authorities: ["<path-to-ca.crt>"]

setup.kibana:
    host: "http://localhost:5601"

```

The output from `sudo metricbeat setup -e d "*"`:

```auto
2021-01-05T12:48:24.063+0100 INFO instance/beat.go:645 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2021-01-05T12:48:24.063+0100 INFO instance/beat.go:653 Beat ID: 4d5b765b-26cb-4050-bed8-416aa5f320e2
2021-01-05T12:48:24.064+0100 INFO [beat] instance/beat.go:981 Beat info {"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "4d5b765b-26cb-4050-bed8-416aa5f320e2"}}}
2021-01-05T12:48:24.064+0100 INFO [beat] instance/beat.go:990 Build info {"system_info": {"build": {"commit": "1428d58cf2ed945441fb2ed03961cafa9e4ad3eb", "libbeat": "7.10.0", "time": "2020-11-09T20:08:47.000Z", "version": "7.10.0"}}}
2021-01-05T12:48:24.064+0100 INFO [beat] instance/beat.go:993 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.14.7"}}}
2021-01-05T12:48:24.065+0100 INFO [beat] instance/beat.go:997 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-11-23T13:52:21+01:00","containerized":false,"name":"<host>","ip":["<ip>","::<more ip>","<ip>","<IPv6>"],"kernel_version":"5.4.0-1029-aws","mac":["<mac>"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.5 LTS (Bionic Beaver)","major":18,"minor":4,"patch":5,"codename":"bionic"},"timezone":"CET","timezone_offset_sec":3600,"id":"ec2cdb77e116a0d46764c463497dcc2f"}}}
2021-01-05T12:48:24.065+0100 INFO [beat] instance/beat.go:1026 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/etc/metricbeat", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 12929, "ppid": 12928, "seccomp": {"mode":"disabled","no_new_privs":false}, "start_time": "2021-01-05T12:48:23.160+0100"}}}
2021-01-05T12:48:24.066+0100 INFO instance/beat.go:299 Setup Beat: metricbeat; Version: 7.10.0
2021-01-05T12:48:24.066+0100 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.10.0' as ILM is enabled.
2021-01-05T12:48:24.066+0100 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"<account>"},"availability_zone":"eu-central-1a","image":{"id":"ami-<ami>"},"instance":{"id":"i-<instance>"},"machine":{"type":<type>"},"provider":"aws","region":"<region>"}
2021-01-05T12:48:24.066+0100 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2021-01-05T12:48:24.067+0100 INFO [publisher] pipeline/module.go:113 Beat name: <....>
2021-01-05T12:48:24.081+0100 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2021-01-05T12:48:24.170+0100 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.10.0
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite: true` for enabling.

2021-01-05T12:48:24.301+0100 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2021-01-05T12:48:24.309+0100 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
2021-01-05T12:48:24.309+0100 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
2021-01-05T12:48:24.309+0100 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{metricbeat-7.10.0 {now/d}-000001}' as ILM is enabled.
2021-01-05T12:48:24.309+0100 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'metricbeat-7.10.0-*' as ILM is enabled.
2021-01-05T12:48:24.309+0100 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {metricbeat-7.10.0 {now/d}-000001} as ILM is enabled.
2021-01-05T12:48:24.309+0100 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {metricbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2021-01-05T12:48:24.319+0100 INFO template/load.go:183 Existing template will be overwritten, as overwrite is enabled.
2021-01-05T12:48:24.600+0100 INFO template/load.go:117 Try loading template metricbeat-7.10.0 to Elasticsearch
2021-01-05T12:48:25.264+0100 INFO template/load.go:109 template with name 'metricbeat-7.10.0' loaded.
2021-01-05T12:48:25.264+0100 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
2021-01-05T12:48:25.272+0100 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
2021-01-05T12:48:25.273+0100 INFO kibana/client.go:119 Kibana url: http://localhost:5601
2021-01-05T12:48:25.708+0100 INFO kibana/client.go:119 Kibana url: http://localhost:5601
2021-01-05T12:50:06.628+0100 INFO instance/beat.go:815 Kibana dashboards successfully loaded.
Loaded dashboards

```

The output from `sudo service metricbeat status`:

```auto
● metricbeat.service - Metricbeat is a lightweight shipper for metrics.
   Loaded: loaded (/lib/systemd/system/metricbeat.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2021-01-05 12:50:37 CET; 3s ago
     Docs: https://www.elastic.co/products/beats/metricbeat
 Main PID: 13015 (metricbeat)
    Tasks: 8 (limit: 4915)
   CGroup: /system.slice/metricbeat.service
           └─13015 /usr/share/metricbeat/bin/metricbeat --environment systemd -c /etc/metricbeat/metricbeat.yml --path.home /usr/share/metricbeat --path.config /etc/metricbeat --path.data /var/lib/metricbeat --path.logs /var/log/metricbeat

Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.177+0100 INFO [beat] instance/beat.go:997 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-11-23T13:52:21+01:00","containerized":false,"n
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.178+0100 INFO [beat] instance/beat.go:1026 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.178+0100 INFO instance/beat.go:299 Setup Beat: metricbeat; Version: 7.10.0
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.178+0100 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.10.0' as ILM is enabled.
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.179+0100 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"<id>
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.179+0100 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.179+0100 INFO [publisher] pipeline/module.go:113 Beat name: <....>
Jan 05 12:50:37 <host> metricbeat[13015]: 2021-01-05T12:50:37.198+0100 WARN [aws.ec2] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
Jan 05 12:50:38 <host> metricbeat[13015]: 2021-01-05T12:50:38.414+0100 WARN [aws.cloudwatch] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
Jan 05 12:50:39 <host> metricbeat[13015]: 2021-01-05T12:50:39.572+0100 WARN [aws.billing] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset

```

But Kibana tells me that: No data has been received from this module yet

And there is no matching index created.

I running out of ideas here and would appreciate any help.

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 5, 2021, 1:11pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/2 "2021-01-05T13:11:32Z")

</div>

Edit:

It seems `sudo metricbeat -e -d "*"` was the command I was looking for. It tells me:

```auto
2021-01-05T13:59:51.818+0100 INFO instance/beat.go:645 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2021-01-05T13:59:51.818+0100 DEBUG [beat] instance/beat.go:697 Beat metadata path: /var/lib/metricbeat/meta.json
2021-01-05T13:59:51.818+0100 INFO instance/beat.go:653 Beat ID: 4d5b765b-26cb-4050-bed8-416aa5f320e2
2021-01-05T13:59:51.818+0100 DEBUG [docker] docker/client.go:48 Docker client will negotiate the API version on the first request.
2021-01-05T13:59:51.819+0100 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:126 add_cloud_metadata: starting to fetch metadata, timeout=3s
2021-01-05T13:59:51.819+0100 DEBUG [add_docker_metadata] add_docker_metadata/add_docker_metadata.go:87 add_docker_metadata: docker environment not detected: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2021-01-05T13:59:51.821+0100 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for gcp after 1.915709ms. result=[provider:gcp, error=failed with http status code 404, metadata={}]
2021-01-05T13:59:51.821+0100 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for digitalocean after 2.281275ms. result=[provider:digitalocean, error=failed with http status code 404, metadata={}]
2021-01-05T13:59:51.822+0100 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:162 add_cloud_metadata: received disposition for aws after 2.675994ms. result=[provider:aws, error=<nil>, metadata={"account":{"id":"<id>"},"availability_zone":"eu-central-1a","image":{"id":"ami-<id>"},"instance":{"id":"i-<id>"},"machine":{"type":"<type>"},"provider":"aws","region":"eu-central-1"}]
2021-01-05T13:59:51.822+0100 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:129 add_cloud_metadata: fetchMetadata ran for 2.812604ms
2021-01-05T13:59:51.822+0100 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"<id>"},"availability_zone":"eu-central-1a","image":{"id":"ami-<id>"},"instance":{"id":"i-<id>"},"machine":{"type":"<type>"},"provider":"aws","region":"eu-central-1"}
2021-01-05T13:59:51.822+0100 DEBUG [processors] processors/processor.go:120 Generated new processors: add_cloud_metadata={"account":{"id":"<id>"},"availability_zone":"eu-central-1a","image":{"id":"ami-<id>"},"instance":{"id":"i-<id>"},"machine":{"type":"<type>"},"provider":"aws","region":"eu-central-1"}, add_docker_metadata=[match_fields=[] match_pids=[process.pid, process.ppid]]
2021-01-05T13:59:51.822+0100 INFO instance/beat.go:392 metricbeat stopped.
2021-01-05T13:59:51.822+0100 ERROR instance/beat.go:956 Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

```

Filebeat is installed but stopped. Just to be sure I added the following to my metricbeat.yml:

```auto
path.home: /usr/share/metricbeat
path.config: /etc/metricbeat
path.data: /var/lib/metricbeat
path.logs: /var/log/

```

But i still get the same error:

````auto
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).```
````

---

<div class="post-metadata">

### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)
#### Post date: [January 5, 2021, 2:11pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/3 "2021-01-05T14:11:39Z")

</div>

Hey!

This means that there is already a Metricbeat process running. I guess it is the systemd service one. You will need to stop Metricbeat service so as to run it manually (`sudo service metricbeat stop`)

C.

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 5, 2021, 3:59pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/4 "2021-01-05T15:59:11Z")

</div>

@ChrsMark thank you for your support.

Although i have stopped and restarted metricbat several times, the error remains the same.

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 7, 2021, 1:43pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/5 "2021-01-07T13:43:15Z")

</div>

The only other \*beat on this machine should be Filebeat, which is currently stopped.

So when I stop Metricbeat manualy, I would expect that no other process is claming the "data path".

But when i start metricbeat with `sudo service metricbeat start` and look at `metricbeat -e -d "*"` I get:

```auto
2021-01-07T14:13:38.072+0100 ERROR instance/beat.go:956 Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

```

Although both beats have there own data path defined:

filebeat.yml:

```auto
filebeat.inputs:
- type: log

  paths:
    - "<path-to->catalina.out"

- type: log
  paths:
    - "<path-to->access.log"
    - "<path-to->error.log"

  multiline.type: pattern
  multiline.pattern: '^[[:space:]]+(at|\.{3})[[:space:]]+\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

- type: filestream
  paths:
    - /var/log/*.log

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.logstash:
  hosts: ["localhost:5144"]

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

path.home: /usr/share/filebeat
path.config: /etc/filebeat
path.data: /var/lib/filebeat
path.logs: /var/log/

```

metricbeat.yml:

```auto
path.home: /usr/share/metricbeat
path.config: /etc/metricbeat
path.data: /var/lib/metricbeat
path.logs: /var/log/

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

metricbeat.modules:
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<pw>'
  secret_access_key: '<pw>'

- module: aws
  period: 300s
  metricsets:
    - cloudwatch
  metrics:
    - namespace: AWS/EC2
      resource_type: ec2:instance
  credential_profile_name: <profile-name>
  access_key_id: '<pw>'
  secret_access_key: '<pw>'

- module: aws
  period: 24h
  metricsets:
    - billing
  access_key_id: '<pw>'
  secret_access_key: '<pw>'
  cost_explorer_config:
    group_by_dimension_keys:
      - "AZ"
      - "INSTANCE_TYPE"
      - "SERVICE"
    group_by_tag_keys:
      - "aws:createdBy"

output.elasticsearch:
  hosts: ["https://localhost:9200"]

```

The only solution that I can think of at the moment is to deinstall Filebeat, but there must be better solution to the problem.

---

<div class="post-metadata">

### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)
#### Post date: [January 7, 2021, 2:52pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/6 "2021-01-07T14:52:44Z")

</div>

When you run `sudo service metricbeat start` is starts a Metricbeat process. When you run `./metricbeat -e -d "*"` then you start another Metricbeat process, you don't just follow the logs of the first Metricbeat. Is this what you might confused you?

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 7, 2021, 3:20pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/7 "2021-01-07T15:20:20Z")

</div>

That makes sense. It eluded me that `./metricbeat -e -d "*"` (of course) is starting a new Metricbeat process.

So `./metricbeat -e -d "*"` now gives the following output:

```auto
...
2021-01-07T15:56:22.777+0100 INFO instance/beat.go:299 Setup Beat: metricbeat; Version: 7.10.0
2021-01-07T15:56:22.777+0100 DEBUG [beat] instance/beat.go:325 Initializing output plugins
2021-01-07T15:56:22.777+0100 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.10.0' as ILM is enabled.
2021-01-07T15:56:22.778+0100 DEBUG [tls] tlscommon/tls.go:172 Successfully loaded CA certificate: <path-to>-ca.crt
2021-01-07T15:56:22.778+0100 INFO eslegclient/connection.go:99 elasticsearch url: https://localhost:9200
2021-01-07T15:56:22.778+0100 DEBUG [publisher] pipeline/consumer.go:148 start pipeline event consumer
2021-01-07T15:56:22.778+0100 INFO [publisher] pipeline/module.go:113 Beat name: abby.yatta.de
2021-01-07T15:56:22.791+0100 DEBUG [modules] beater/metricbeat.go:151 Available modules and metricsets: Register [...]
2021-01-07T15:56:22.791+0100 DEBUG [get_aws_credentials] aws/credentials.go:40 Using access_key_id, secret_access_key and/or session_token for AWS credential
2021-01-07T15:56:22.792+0100 DEBUG [aws.ec2] aws/aws.go:97 Metricset level config for period: 5m0s
2021-01-07T15:56:22.792+0100 DEBUG [aws.ec2] aws/aws.go:98 Metricset level config for tags filter: []
2021-01-07T15:56:22.792+0100 WARN [aws.ec2] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
2021-01-07T15:56:23.228+0100 DEBUG [aws.ec2] aws/aws.go:121 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:23.604+0100 DEBUG [aws.ec2] aws/aws.go:176 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:24.007+0100 DEBUG [aws.ec2] aws/aws.go:139 Metricset level config for regions: [...]
2021-01-07T15:56:24.008+0100 DEBUG [get_aws_credentials] aws/credentials.go:40 Using access_key_id, secret_access_key and/or session_token for AWS credential
2021-01-07T15:56:24.008+0100 DEBUG [aws.cloudwatch] aws/aws.go:97 Metricset level config for period: 5m0s
2021-01-07T15:56:24.008+0100 DEBUG [aws.cloudwatch] aws/aws.go:98 Metricset level config for tags filter: []
2021-01-07T15:56:24.008+0100 WARN [aws.cloudwatch] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
2021-01-07T15:56:24.381+0100 DEBUG [aws.cloudwatch] aws/aws.go:121 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:24.758+0100 DEBUG [aws.cloudwatch] aws/aws.go:176 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:25.157+0100 DEBUG [aws.cloudwatch] aws/aws.go:139 Metricset level config for regions: [...]
2021-01-07T15:56:25.157+0100 DEBUG [cloudwatch] cloudwatch/cloudwatch.go:127 cloudwatch config = {[{AWS/EC2 [] [] ec2:instance [] []}]}
2021-01-07T15:56:25.157+0100 DEBUG [get_aws_credentials] aws/credentials.go:40 Using access_key_id, secret_access_key and/or session_token for AWS credential
2021-01-07T15:56:25.157+0100 DEBUG [aws.billing] aws/aws.go:97 Metricset level config for period: 24h0m0s
2021-01-07T15:56:25.157+0100 DEBUG [aws.billing] aws/aws.go:98 Metricset level config for tags filter: []
2021-01-07T15:56:25.157+0100 WARN [aws.billing] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
2021-01-07T15:56:25.528+0100 DEBUG [aws.billing] aws/aws.go:121 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:25.905+0100 DEBUG [aws.billing] aws/aws.go:176 AWS Credentials belong to account ID: <id>
2021-01-07T15:56:26.306+0100 DEBUG [aws.billing] aws/aws.go:139 Metricset level config for regions: [eu-north-1 ap-south-1 eu-west-3 eu-west-2 eu-west-1 ap-northeast-2 ap-northeast-1 sa-east-1 ca-central-1 ap-southeast-1 ap-southeast-2 eu-central-1 us-east-1 us-east-2 us-west-1 us-west-2]
2021-01-07T15:56:26.306+0100 DEBUG [billing] billing/billing.go:91 cost explorer config = {{[AZ INSTANCE_TYPE SERVICE] [aws:createdBy]}}
2021-01-07T15:56:26.306+0100 INFO instance/beat.go:455 metricbeat start running.
2021-01-07T15:56:26.306+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=aws, len(metricSetWrappers)=1]
2021-01-07T15:56:26.306+0100 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2021-01-07T15:56:26.306+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=aws, len(metricSetWrappers)=1]
2021-01-07T15:56:26.306+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=aws, len(metricSetWrappers)=1]
2021-01-07T15:56:26.307+0100 DEBUG [cfgfile] cfgfile/reload.go:132 Checking module configs from: /etc/metricbeat/modules.d/*.yml
2021-01-07T15:56:26.306+0100 DEBUG [module] module/wrapper.go:181 aws/ec2 will start after 9.512790174s
2021-01-07T15:56:26.307+0100 DEBUG [module] module/wrapper.go:181 aws/cloudwatch will start after 7.01882229s
2021-01-07T15:56:26.307+0100 DEBUG [module] module/wrapper.go:181 aws/billing will start after 6.509615922s
2021-01-07T15:56:26.307+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-01-07T15:56:26.307+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/system.yml
2021-01-07T15:56:26.307+0100 DEBUG [cfgfile] cfgfile/reload.go:146 Number of module configs found: 9
2021-01-07T15:56:26.308+0100 DEBUG [get_aws_credentials] aws/credentials.go:74 Using shared credential profile for AWS credential
2021-01-07T15:56:26.312+0100 INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"cfs":{"period":{"us":100000}},"id":"user.slice"},"cpuacct":{"id":"user.slice","total":{"ns":14732106278}},"memory":{"id":"user.slice","mem":{"limit":{"bytes":9223372036854771712},"usage":{"bytes":231477248}}}},"cpu":{"system":{"ticks":30,"time":{"ms":34}},"total":{"ticks":270,"time":{"ms":276},"value":270},"user":{"ticks":240,"time":{"ms":242}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":17},"info":{"ephemeral_id":"aaec70ef-d934-4054-851a-4ad733f2757f","uptime":{"ms":3612}},"memstats":{"gc_next":18309104,"memory_alloc":13727312,"memory_total":41730768,"rss":85676032},"runtime":{"goroutines":49}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":3,"events":{"active":0}}},"system":{"cpu":{"cores":4},"load":{"1":0.15,"15":0.12,"5":0.17,"norm":{"1":0.0375,"15":0.03,"5":0.0425}}}}}}
2021-01-07T15:56:26.312+0100 INFO [monitoring] log/log.go:154 Uptime: 3.613754169s
2021-01-07T15:56:26.312+0100 INFO [monitoring] log/log.go:131 Stopping metrics logging.
2021-01-07T15:56:26.312+0100 INFO instance/beat.go:461 metricbeat stopped.
2021-01-07T15:56:26.312+0100 ERROR instance/beat.go:956 Exiting: 6 errors: metricset 'aws/elb' not found; metricset 'aws/natgateway' not found; error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make Client request
caused by: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>
; metricset 'aws/transitgateway' not found; metricset 'aws/usage' not found; metricset 'aws/vpn' not found
Exiting: 6 errors: metricset 'aws/elb' not found; metricset 'aws/natgateway' not found; error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make Client request
caused by: <?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>
; metricset 'aws/transitgateway' not found; metricset 'aws/usage' not found; metricset 'aws/vpn' not found

```

Most important, I think, is:

```auto
error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found

```

My AWS Admin told me that my account should now have the following AWS permissions:

ec2:DescribeInstances  
ec2:DescribeRegions  
cloudwatch:GetMetricData  
cloudwatch:ListMetrics  
sts:GetCallerIdentity  
iam:ListAccountAliases  
tag:getResources  
sts:GetCallerIdentity  
ce:GetCostAndUsage

Which should be necessary permissions for the metricsetzs: ec2, cloudwatch and billing. Or am I mistaken?

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [January 7, 2021, 3:45pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/8 "2021-01-07T15:45:06Z")

</div>

Hi @Jurilz 👋 Looking at your config, the ec2 metricset part and cloudwatch metricset part are basically doing the same thing so you are actually trying to collect ec2 metrics twice with this config. Also in your config, you have `metricbeat.config.modules` section to specify `path`. That means the config Metricbeat will look is `modules.d/aws.yml`. Did you run `./metricbeat modules enable aws`? If so, what does `modules.d/aws.yml` look like?

For better eliminating the problem, could you try only with the ec2 metricset section in the config please? TIA!!

```auto
metricbeat.modules:
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<pw>'
  secret_access_key: '<pw>'

```

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 7, 2021, 5:15pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/9 "2021-01-07T17:15:06Z")

</div>

Hi @Kaiyan_Sheng, thank you for your help.

Thank you also for the tipp, that the metricsets `ec2` and `cloudwatch` are doing the same thing here. I was unsure about the difference between them, so I just put both in.

aws module should be enabled. `./metricbeat modules aws` gives me:  
`Module aws is already enabled`

the `modules.d/aws.yml` was probably part of the problem, because I never changed it so it stayed in the default state.

So I changed it to `modules.d/aws.yml`:

```auto
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<key-id>'
  secret_access_key: '<secret-key>'

```

I also followed your suggestion and changed `metricbeat.yml` to :

```auto
path.home: /usr/share/metricbeat
path.config: /etc/metricbeat
path.data: /var/lib/metricbeat
path.logs: /var/log/

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

metricbeat.modules:
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<key>'
  secret_access_key: '<secret-key>'

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: elastic
  password: <pw>
  ssl.certificate_authorities: ["<path-to>-ca.crt"]

setup.kibana:
    host: "http://localhost:5601"

```

I get that the definiton of the aws module is duplicated at this point, so I should probably take it out from the `metricbeat.yml`.

I still don't get any data from Metricbeat, but `./metricbeat -e -d "*"` gives me never stopping output like:

```auto
2021-01-07T18:11:07.906+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_license?human=false <nil>
2021-01-07T18:11:07.993+0100 DEBUG [license] licenser/check.go:31 Checking that license covers %sBasic
2021-01-07T18:11:07.993+0100 INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic
2021-01-07T18:11:07.993+0100 DEBUG [esclientleg] eslegclient/connection.go:290 ES Ping(url=https://localhost:9200)
2021-01-07T18:11:08.002+0100 DEBUG [esclientleg] eslegclient/connection.go:313 Ping status code: 200
2021-01-07T18:11:08.002+0100 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.10.0
2021-01-07T18:11:08.002+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_xpack <nil>
2021-01-07T18:11:08.087+0100 INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2021-01-07T18:11:08.087+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_ilm/policy/metricbeat <nil>
2021-01-07T18:11:08.096+0100 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
2021-01-07T18:11:08.096+0100 INFO [index-management] idxmgmt/std.go:274 ILM policy successfully loaded.
2021-01-07T18:11:08.096+0100 INFO [index-management] idxmgmt/std.go:407 Set setup.template.name to '{metricbeat-7.10.0 {now/d}-000001}' as ILM is enabled.
2021-01-07T18:11:08.096+0100 INFO [index-management] idxmgmt/std.go:412 Set setup.template.pattern to 'metricbeat-7.10.0-*' as ILM is enabled.
2021-01-07T18:11:08.096+0100 INFO [index-management] idxmgmt/std.go:446 Set settings.index.lifecycle.rollover_alias in template to {metricbeat-7.10.0 {now/d}-000001} as ILM is enabled.
2021-01-07T18:11:08.097+0100 INFO [index-management] idxmgmt/std.go:450 Set settings.index.lifecycle.name in template to {metricbeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
2021-01-07T18:11:08.097+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_cat/templates/metricbeat-7.10.0 <nil>
2021-01-07T18:11:08.106+0100 INFO template/load.go:97 Template metricbeat-7.10.0 already exists and will not be overwritten.
2021-01-07T18:11:08.106+0100 INFO [index-management] idxmgmt/std.go:298 Loaded index template.
2021-01-07T18:11:08.106+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_alias/metricbeat-7.10.0 <nil>
2021-01-07T18:11:08.115+0100 INFO [index-management] idxmgmt/std.go:309 Write alias successfully generated.
2021-01-07T18:11:08.115+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/ <nil>
2021-01-07T18:11:08.124+0100 INFO [publisher_pipeline_output] pipeline/output.go:151 Connection to backoff(elasticsearch(https://localhost:9200)) established
2021-01-07T18:11:08.170+0100 DEBUG [elasticsearch] elasticsearch/client.go:230 PublishEvents: 21 events have been published to elasticsearch in 46.007978ms.
2021-01-07T18:11:08.171+0100 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [0: 0, 21]
2021-01-07T18:11:08.171+0100 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:21
2021-01-07T18:11:08.171+0100 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack
2021-01-07T18:11:11.746+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=aws, name=ec2, host=]
2021-01-07T18:11:11.746+0100 DEBUG [aws.ec2] ec2/ec2.go:92 startTime = 2021-01-07 18:01:11.746275539 +0100 CET m=-591.274983020, endTime = 2021-01-07 18:11:11.746275539 +0100 CET m=+8.725016980

...

    "ppid": 1176,
    "pgid": 4635,
    "working_directory": "/var/lib/postgresql/10/main"
  },
  "event": {
    "duration": 47187242,
    "dataset": "system.process",
    "module": "system"
  },
  "metricset": {
    "name": "process",
    "period": 10000
  },
  "ecs": {
    "version": "1.6.0"
  },
  "agent": {
    "type": "metricbeat",
    "version": "7.10.0",
    "hostname": "<host>",
    "ephemeral_id": "<id>",
    "id": "<id>",
    "name": "<name>"
  }
}
2021-01-07T18:11:17.901+0100 DEBUG [elasticsearch] elasticsearch/client.go:230 PublishEvents: 16 events have been published to elasticsearch in 46.068205ms.
2021-01-07T18:11:17.901+0100 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [1: 0, 16]
2021-01-07T18:11:17.901+0100 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:16
2021-01-07T18:11:17.901+0100 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack

```

It just sends or tries to send data and counts up on `2021-01-07T18:11:08.171+0100 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [0: 0, 21]`

When I start metricbeat after that, the status is running, but elasticsearch and kibana still don't receive any data.

---

<div class="post-metadata">

### Author: ![bplies](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bplies/32/81481_2.png) [@bplies](https://discuss.elastic.co/u/bplies)
#### Post date: [January 7, 2021, 8:37pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/10 "2021-01-07T20:37:20Z")

</div>

We've got the same issue going on right now. If we change the output to local filesystem we can see the data from AWS Cloudwatch. Once we switch the output to Elasticsearch, it never receives anything. Elastic Cloud logs don't show any issue. Metricbeat logs do not show any issue.

version: 7.10.1

---

<div class="post-metadata">

### Author: ![bplies](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bplies/32/81481_2.png) [@bplies](https://discuss.elastic.co/u/bplies)
#### Post date: [January 7, 2021, 10:11pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/11 "2021-01-07T22:11:25Z")

</div>

A quick update. There seems to have been a similarly-described issue reported on [Github](https://github.com/elastic/beats/issues/21437) for Metricbeat 7.9.2 but was closed.

We disabled the AWS module and enabled the System module and that was able to write at least into Elastic - confirming that our output is defined correctly and working.

Another interesting note is that CTRL+C doesn't work to kill Metricbeat when the AWS module was enabled and it needed an OS-level process kill. With System module, however, CTRL+C works and the Metricbeat logs routinely confirm publishing events to Elasticsearch.

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [January 7, 2021, 11:15pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/12 "2021-01-07T23:15:49Z")

</div>

Thank you! Now you have `metricbeat.yml` points to modules.d to get config, let's remove the `metricbeat.modules` section in `metricbeat.yml`:

```auto
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<key>'
  secret_access_key: '<secret-key>'

```

(delete this part in metricbeat.yml)

Also could you add `logging.level: debug` in `metricbeat.yml` please? This way we can see some debug messages, maybe it will help.

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [January 7, 2021, 11:25pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/13 "2021-01-07T23:25:27Z")

</div>

Thanks @bplies! Are you using Elasticsearch on Elastic Cloud? I will try to reproduce this.

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 8, 2021, 12:30pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/14 "2021-01-08T12:30:22Z")

</div>

Thank you for the tipp. I added `logging.level: debug` and removed the aws module from `metricbeat.yml`:

```auto
path.home: /usr/share/metricbeat
path.config: /etc/metricbeat
path.data: /var/lib/metricbeat
path.logs: /var/log/

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: elastic
  password: <pw>
  ssl.certificate_authorities: ["<path-to>-ca.crt"]

setup.kibana:
    host: "http://localhost:5601"

logging.level: debug

```

modules.d/aws.yml:

```auto
- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: '<key>'
  secret_access_key: '<secret-key>'

```

`metricbeat -e -d "*"` gives me the following output:

```auto
2021-01-08T13:02:40.082+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-01-08T13:02:40.082+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/system.yml
2021-01-08T13:02:40.083+0100 DEBUG [cfgfile] cfgfile/reload.go:146 Number of module configs found: 4
2021-01-08T13:02:40.083+0100 DEBUG [get_aws_credentials] aws/credentials.go:40 Using access_key_id, secret_access_key and/or session_token for AWS credential
2021-01-08T13:02:40.083+0100 DEBUG [aws.ec2] aws/aws.go:97 Metricset level config for period: 5m0s
2021-01-08T13:02:40.083+0100 DEBUG [aws.ec2] aws/aws.go:98 Metricset level config for tags filter: []
2021-01-08T13:02:40.083+0100 WARN [aws.ec2] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
2021-01-08T13:02:40.507+0100 DEBUG [aws.ec2] aws/aws.go:121 AWS Credentials belong to account ID: <id>
2021-01-08T13:02:40.890+0100 DEBUG [aws.ec2] aws/aws.go:176 AWS Credentials belong to account ID: <id>
2021-01-08T13:02:41.478+0100 DEBUG [aws.ec2] aws/aws.go:139 Metricset level config for regions: [...]
2021-01-08T13:02:41.479+0100 DEBUG [system.process] process/process.go:86 process cgroup data collection is enabled, using hostfs=''
2021-01-08T13:02:41.479+0100 INFO filesystem/filesystem.go:57 Ignoring filesystem types: sysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, ecryptfs, fuse, fusectl, mqueue, pstore, autofs
2021-01-08T13:02:41.479+0100 INFO [system.fsstat] fsstat/fsstat.go:57 Ignoring filesystem types: %ssysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, ecryptfs, fuse, fusectl, mqueue, pstore, autofs
2021-01-08T13:02:41.480+0100 INFO cfgfile/reload.go:164 Config reloader started
2021-01-08T13:02:41.480+0100 DEBUG [cfgfile] cfgfile/reload.go:194 Scan for new config files
2021-01-08T13:02:41.480+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-01-08T13:02:41.480+0100 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/system.yml
2021-01-08T13:02:41.480+0100 DEBUG [cfgfile] cfgfile/reload.go:213 Number of module configs found: 4
2021-01-08T13:02:41.480+0100 DEBUG [reload] cfgfile/list.go:63 Starting reload procedure, current runners: 0
2021-01-08T13:02:41.481+0100 DEBUG [reload] cfgfile/list.go:81 Start list: 4, Stop list: 0
2021-01-08T13:02:41.481+0100 DEBUG [get_aws_credentials] aws/credentials.go:40 Using access_key_id, secret_access_key and/or session_token for AWS credential
2021-01-08T13:02:41.481+0100 DEBUG [aws.ec2] aws/aws.go:97 Metricset level config for period: 5m0s
2021-01-08T13:02:41.481+0100 DEBUG [aws.ec2] aws/aws.go:98 Metricset level config for tags filter: []
2021-01-08T13:02:41.481+0100 WARN [aws.ec2] aws/aws.go:99 extra charges on AWS API requests will be generated by this metricset
2021-01-08T13:02:41.853+0100 DEBUG [aws.ec2] aws/aws.go:121 AWS Credentials belong to account ID: <id>
2021-01-08T13:02:42.228+0100 DEBUG [aws.ec2] aws/aws.go:176 AWS Credentials belong to account ID: <id>
2021-01-08T13:02:42.635+0100 DEBUG [aws.ec2] aws/aws.go:139 Metricset level config for regions: [...]
2021-01-08T13:02:42.635+0100 DEBUG [reload] cfgfile/list.go:105 Starting runner: RunnerGroup{aws [metricsets=1]}
2021-01-08T13:02:42.636+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=aws, len(metricSetWrappers)=1]
2021-01-08T13:02:42.636+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=aws, name=ec2, host=]
2021-01-08T13:02:42.637+0100 DEBUG [system.process] process/process.go:86 process cgroup data collection is enabled, using hostfs=''
2021-01-08T13:02:42.637+0100 DEBUG [aws.ec2] ec2/ec2.go:92 startTime = 2021-01-08 12:52:42.637116494 +0100 CET m=-597.345239644, endTime = 2021-01-08 13:02:42.637116494 +0100 CET m=+2.654760356
2021-01-08T13:02:42.638+0100 DEBUG [reload] cfgfile/list.go:105 Starting runner: RunnerGroup{system [metricsets=1], system [metricsets=1], system [metricsets=1], system [metricsets=1], system [metricsets=1], system [metricsets=1], system [metricsets=1]}
2021-01-08T13:02:42.639+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=system, len(metricSetWrappers)=1]
...
2021-01-08T13:02:42.639+0100 DEBUG [module] module/wrapper.go:127 Starting Wrapper[name=system, len(metricSetWrappers)=1]
2021-01-08T13:02:42.639+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=system, name=load, host=]
...
2021-01-08T13:02:42.639+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=system, name=process_summary, host=]
2021-01-08T13:02:42.639+0100 INFO filesystem/filesystem.go:57 Ignoring filesystem types: sysfs, tmpfs, bdev, proc, cgroup, cgroup2, cpuset, devtmpfs, configfs, debugfs, tracefs, securityfs, sockfs, bpf, pipefs, ramfs, hugetlbfs, devpts, ecryptfs, fuse, fusectl, mqueue, pstore, autofs
2021-01-08T13:02:42.640+0100 DEBUG [processors] processing/processors.go:203 Publish event: {...}
...
2021-01-08T13:02:42.650+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=system, name=fsstat, host=]
2021-01-08T13:02:42.659+0100 DEBUG [system.fsstat] fsstat/fsstat.go:87 filesystem: /var/lib/lxcfs total=0, used=0, free=0
2021-01-08T13:02:42.659+0100 DEBUG [system.fsstat] fsstat/fsstat.go:87 filesystem: / total=312201752576, used=60525809664, free=251675942912
2021-01-08T13:02:42.650+0100 DEBUG [module] module/wrapper.go:189 Starting metricSetWrapper[module=system, name=uptime, host=]
2021-01-08T13:02:42.660+0100 DEBUG [publisher] pipeline/client.go:231 Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:2021-01-08 13:02:42.650010207 +0100 CET m=+2.667654210 Meta:null Fields:{"event":{"dataset":"system.filesystem","duration":10239381,"module":"system"},"metricset":{"name":"filesystem","period":60000},"service":{"type":"system"},"system":{"filesystem":{"available":0,"device_name":"/dev/loop2","files":10809,"free":0,"free_files":0,"mount_point":"/snap/core18/1944","total":58195968,"type":"squashfs","used":{"bytes":58195968,"pct":1.000000}}}} Private:<nil> TimeSeries:true}
2021-01-08T13:02:42.660+0100 DEBUG [publisher] pipeline/client.go:231 Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:2021-01-08 13:02:42.650010207 +0100 CET m=+2.667654210 Meta:null Fields:{"event":{"dataset":"system.filesystem","duration":10281937,"module":"system"},"metricset":{"name":"filesystem","period":60000},"service":{"type":"system"},"system":{"filesystem":{"available":0,"device_name":"/dev/loop3","files":15,"free":0,"free_files":0,"mount_point":"/snap/amazon-ssm-agent/2333","total":29491200,"type":"squashfs","used":{"bytes":29491200,"pct":1.000000}}}} Private:<nil> TimeSeries:true}
...
2021-01-08T13:02:42.661+0100 DEBUG [publisher] pipeline/client.go:231 Pipeline client receives callback 'onFilteredOut' for event: {Timestamp:2021-01-08 13:02:42.650010207 +0100 CET m=+2.667654210 Meta:null Fields:{"event":{"dataset":"system.filesystem","duration":10964282,"module":"system"},"metricset":{"name":"filesystem","period":60000},"service":{"type":"system"},"system":{"filesystem":{"available":0,"device_name":"/dev/loop5","files":10779,"free":0,"free_files":0,"mount_point":"/snap/core18/1932","total":58064896,"type":"squashfs","used":{"bytes":58064896,"pct":1.000000}}}} Private:<nil> TimeSeries:true}
2021-01-08T13:02:42.661+0100 DEBUG [processors] processing/processors.go:203 Publish event: {...}
...
2021-01-08T13:05:33.500+0100 DEBUG [elasticsearch] elasticsearch/client.go:230 PublishEvents: 16 events have been published to elasticsearch in 69.441716ms.
2021-01-08T13:05:33.500+0100 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [1: 0, 16]
2021-01-08T13:05:33.501+0100 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:16
2021-01-08T13:05:33.501+0100 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack
...
2021-01-08T13:06:52.541+0100 DEBUG [processors] processing/processors.go:203 Publish event: {
  "@timestamp": "2021-01-08T12:06:52.495Z",
  "@metadata": {
    "beat": "metricbeat",
    "type": "_doc",
    "version": "7.10.0"
  },
  "agent": {
    "ephemeral_id": "<id>",
    "id": "<id>",
    "name": "<name>",
    "type": "metricbeat",
    "version": "7.10.0",
    "hostname": "<host>"
  },
  "ecs": {
    "version": "1.6.0"
  },
  "service": {
    "type": "system"
  },
  "system": {
    "process": {
      "cmdline": "<db>: 10/main: <process> idle ",
      "cgroup": {
        "path": "/...",
        "cpu": {
          "path": "/...",
          "cfs": {
            "quota": {
              "us": 0
            },
            "shares": 1024,
            "period": {
              "us": 100000
            }
          },
          "rt": {
            "period": {
              "us": 0
            },
            "runtime": {
              "us": 0
            }
          },
          "stats": {...},
          "id": "<id>"
        },
        "cpuacct": {
          "percpu": {...},
          "id": "<id>",
          "path": "/...",
          "total": {
            "ns": 54382023346
          },
          "stats": {...}
        },
        "memory": {
          "kmem_tcp": {...},
            "usage": {...}
          },
          "stats": {...},
          "id": "<id>",
          "path": "/...",
          "mem": {
            "failures": 0,
            "limit": {
              "bytes": 9223372036854771712
            },
            "usage": {
              "bytes": 271331328,
              "max": {
                "bytes": 275591168
              }
            }
          },
          "memsw": {...},
          "kmem": {
            "failures": 0,
            "limit": {
              "bytes": 9223372036854771712
            },
            "usage": {...}
          }
        },
        "blkio": {...},
        "id": "<id>"
      },
      "memory": {...},
      "cpu": {
        "total": {...},
        "start_time": "2021-01-07T12:54:40.000Z"
      },
      "fd": {...},
      "state": "sleeping"
    }
  },
  "event": {
    "dataset": "system.process",
    "module": "system",
    "duration": 44376794
  },
  "host": {
    "name": "<name>"
  },
  "cloud": {
    "region": "...",
    "provider": "aws",
    "availability_zone": "<zone>",
    "account": {
      "id": "<id>"
    },
    "image": {
      "id": "ami-<id>"
    },
    "instance": {
      "id": "i-<id>"
    },
    "machine": {
      "type": "<type>"
    }
  },
  "metricset": {
    "period": 10000,
    "name": "process"
  },
  "process": {...},
  "user": {
    "name": "<name>"
  }
}
...
2021-01-08T13:06:13.484+0100 DEBUG [elasticsearch] elasticsearch/client.go:230 PublishEvents: 16 events have been published to elasticsearch in 52.976221ms.
2021-01-08T13:06:13.484+0100 DEBUG [publisher] memqueue/ackloop.go:160 ackloop: receive ack [5: 0, 16]
2021-01-08T13:06:13.484+0100 DEBUG [publisher] memqueue/ackloop.go:128 ackloop: return ack to broker loop:16
2021-01-08T13:06:13.484+0100 DEBUG [publisher] memqueue/ackloop.go:131 ackloop: done send ack

```

and keeps going on like that.

---

<div class="post-metadata">

### Author: ![bplies](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bplies/32/81481_2.png) [@bplies](https://discuss.elastic.co/u/bplies)
#### Post date: [January 8, 2021, 4:51pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/15 "2021-01-08T16:51:59Z")

</div>

Yes. On Elastic Cloud.

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 9, 2021, 1:37pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/16 "2021-01-09T13:37:50Z")

</div>

It seems that there could be a connection problem from metricbeat to elasticsearch.

This is the output from `sudo service metricbeat status` after starting metricbeat with the new loglevel (debug):

```auto
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.031+0100 DEBUG [esclientleg] eslegclient/connection.go:290 ES Ping(url=https://localhost:9200)
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.142+0100 DEBUG [esclientleg] eslegclient/connection.go:313 Ping status code: 200
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.142+0100 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.10.0
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.142+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_license?human=false <nil>
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.228+0100 DEBUG [license] licenser/check.go:31 Checking that license covers %sBasic
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.228+0100 INFO [license] licenser/es_callback.go:51 Elasticsearch license: Basic
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.228+0100 DEBUG [esclientleg] eslegclient/connection.go:290 ES Ping(url=https://localhost:9200)
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.236+0100 DEBUG [esclientleg] eslegclient/connection.go:313 Ping status code: 200
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.236+0100 INFO [esclientleg] eslegclient/connection.go:314 Attempting to connect to Elasticsearch version 7.10.0
Jan 09 14:29:05 <host> metricbeat[24560]: 2021-01-09T14:29:05.236+0100 DEBUG [esclientleg] eslegclient/connection.go:364 GET https://localhost:9200/_xpack <nil>

```

especially the `GET https://localhost:9200/_xpack <nil>` somehow worries me.

This is my elasticsearch.yml:

```auto
network.host: 0.0.0.0

# security settings
xpack.security.enabled: true

# transportationlayer
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.certificate_authorities: ["<path-to>-ca.crt"]
xpack.security.transport.ssl.certificate: "<path-to>-elasticsearch.crt"
xpack.security.transport.ssl.key: "<path-to>-elasticsearch.key"

# This turns on SSL for the HTTP (Rest) interface
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.certificate_authorities: ["<path-to>-ca.crt"]
xpack.security.http.ssl.keystore.path: "<path-to>-http.p12"
xpack.security.http.ssl.keystore.password: "<pw>"

```

---

<div class="post-metadata">

### Author: ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)
#### Post date: [January 11, 2021, 9:07am UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/17 "2021-01-11T09:07:33Z")

</div>

@Jurilz Do you see any errors? Could you describe more about what this possible connection problem? Do you see any unsuccessful requests? Any errors on Elasticsearch's logs? Also please make sure your configuration is not missing any pieces about [Secure communication with Elasticsearch](https://www.elastic.co/guide/en/beats/metricbeat/current/securing-communication-elasticsearch.html#securing-communication-elasticsearch).

---

<div class="post-metadata">

### Author: ![Jurilz](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@Jurilz](https://discuss.elastic.co/u/Jurilz)
#### Post date: [January 11, 2021, 11:45am UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/18 "2021-01-11T11:45:46Z")

</div>

Elasticsearch now successfully receives data from Metricbeat. 😀

Although the data is not the default data that the Kibana Dashboard is expecting, but I would think this is a topic for another thread.

I did to things, that could made the difference:

1. There was an old metricbeat index in elasticsearch and I deleted it manually.

2. I added the username and password to the the kibana host configuration.

my metricbeat.yml:

```auto
path.home: /usr/share/metricbeat
path.config: /etc/metricbeat
path.data: /var/lib/metricbeat
path.logs: /var/log/

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: elastic
  password: <pw>
  ssl.certificate_authorities: ["<path-to>-ca.crt"]

setup.kibana:
  host: "http://localhost:5601"
  username: elastic
  password: <pw>

```

Thank you again @ChrsMark and @Kaiyan_Sheng for your help.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 8, 2021, 1:45pm UTC](https://discuss.elastic.co/t/metricbeat-aws-module-not-sending-data/260200/19 "2021-02-08T13:45:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.