# Aws module

**URL:** <https://discuss.elastic.co/t/aws-module/197618>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [September 1, 2019, 10:43am UTC](https://discuss.elastic.co/t/aws-module/197618 "2019-09-01T10:43:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nee\_Defeng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nee_defeng/32/45771_2.png) [@Nee\_Defeng](https://discuss.elastic.co/u/Nee_Defeng)\
**Post date:** [September 1, 2019, 10:43am UTC](https://discuss.elastic.co/t/aws-module/197618/1 "2019-09-01T10:43:20Z")

</div>

There was no error in the debug log, and there was no events pulled from AWS (in metricbeat-\* index there was no message).

Please help, thanks in advance!

Here is the config:

```
# Module: aws
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.3/metricbeat-module-aws.html

- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
- module: aws
  period: 300s
  metricsets:
    - sqs
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
  #regions:
  # - us-west-1
- module: aws
  period: 86400s
  metricsets:
    - s3_request
    - s3_daily_storage
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
  #regions:
  # - us-west-1

```

Here is the debug log:

[root@elk ~]# metricbeat -e -d "\*"  
2019-09-01T06:36:51.932-0400 INFO instance/beat.go:606 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]  
2019-09-01T06:36:51.932-0400 DEBUG [beat] instance/beat.go:658 Beat metadata path: /var/lib/metricbeat/meta.json  
2019-09-01T06:36:51.932-0400 INFO instance/beat.go:614 Beat ID: cb4997de-b548-4ea4-9a48-9ab8fbb9dabe  
2019-09-01T06:36:51.935-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:164 add\_cloud\_metadata: starting to fetch metadata, timeout=3s  
2019-09-01T06:36:51.948-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for gcp after 12.560124ms. result=[provider:gcp, error=failed requesting gcp metadata: Get [http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json:](http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json:) dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]  
2019-09-01T06:36:51.948-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for openstack after 12.731171ms. result=[provider:openstack, error=failed requesting openstack metadata: Get [http://169.254.169.254/2009-04-04/meta-data/instance-id:](http://169.254.169.254/2009-04-04/meta-data/instance-id:) dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]  
2019-09-01T06:36:51.948-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for aws after 12.812023ms. result=[provider:aws, error=failed requesting aws metadata: Get [http://169.254.169.254/2014-02-25/dynamic/instance-identity/document:](http://169.254.169.254/2014-02-25/dynamic/instance-identity/document:) dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]  
2019-09-01T06:36:51.948-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for digitalocean after 12.964448ms. result=[provider:digitalocean, error=failed requesting digitalocean metadata: Get [http://169.254.169.254/metadata/v1.json:](http://169.254.169.254/metadata/v1.json:) dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]  
2019-09-01T06:36:51.950-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for az after 14.051429ms. result=[provider:az, error=failed requesting az metadata: Get [http://169.254.169.254/metadata/instance/compute?api-version=2017-04-02:](http://169.254.169.254/metadata/instance/compute?api-version=2017-04-02:) dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]  
2019-09-01T06:36:52.558-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for qcloud after 622.875441ms. result=[provider:qcloud, error=failed requesting qcloud metadata: Get [http://metadata.tencentyun.com/meta-data/instance-id:](http://metadata.tencentyun.com/meta-data/instance-id:) dial tcp: lookup [metadata.tencentyun.com](http://metadata.tencentyun.com) on 192.168.245.2:53: no such host, metadata={}]  
2019-09-01T06:36:54.937-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:196 add\_cloud\_metadata: received disposition for ecs after 3.001561093s. result=[provider:ecs, error=failed requesting ecs metadata: Get [http://100.100.100.200/latest/meta-data/region-id:](http://100.100.100.200/latest/meta-data/region-id:) net/http: request canceled (Client.Timeout exceeded while awaiting headers), metadata={}]  
2019-09-01T06:36:54.937-0400 DEBUG [filters] add\_cloud\_metadata/add\_cloud\_metadata.go:167 add\_cloud\_metadata: fetchMetadata ran for 3.001629667s  
2019-09-01T06:36:54.937-0400 INFO add\_cloud\_metadata/add\_cloud\_metadata.go:347 add\_cloud\_metadata: hosting provider type not detected.  
2019-09-01T06:36:54.937-0400 DEBUG [processors] processors/processor.go:93 Generated new processors: add\_host\_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]], add\_cloud\_metadata=null  
2019-09-01T06:36:54.937-0400 INFO [seccomp] seccomp/seccomp.go:101 Syscall filter could not be installed because the kernel does not support seccomp

---

<div class="post-metadata">

**Author:** ![Nee\_Defeng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nee_defeng/32/45771_2.png) [@Nee\_Defeng](https://discuss.elastic.co/u/Nee_Defeng)\
**Post date:** [September 1, 2019, 10:43am UTC](https://discuss.elastic.co/t/aws-module/197618/2 "2019-09-01T10:43:52Z")

</div>

2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:902 Beat info {"system\_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "cb4997de-b548-4ea4-9a48-9ab8fbb9dabe"}}}  
2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:911 Build info {"system\_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:33:03.000Z", "version": "7.3.1"}}}  
2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:914 Go runtime info {"system\_info": {"go": {"os":"linux","arch":"amd64","max\_procs":8,"version":"go1.12.4"}}}  
2019-09-01T06:36:54.939-0400 INFO [beat] instance/beat.go:918 Host info {"system\_info": {"host": {"architecture":"x86\_64","boot\_time":"2019-09-01T04:54:26-04:00","containerized":false,"name":"elk.tongyao.com.au","ip":["127.0.0.1/8","::1/128","192.168.245.129/24","fe80::6e39:8fcb:710:ee1a/64"],"kernel\_version":"3.10.0-693.el7.x86\_64","mac":["00:0c:29:a9:9b:c4"],"os":{"family":"redhat","platform":"rhel","name":"Red Hat Enterprise Linux Server","version":"7.4 (Maipo)","major":7,"minor":4,"patch":0,"codename":"Maipo"},"timezone":"EDT","timezone\_offset\_sec":-14400,"id":"a43e1577a5804785a131ba4d1ef34ea3"}}}  
2019-09-01T06:36:54.939-0400 INFO [beat] instance/beat.go:947 Process info {"system\_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend"],"effective":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend"],"bounding":["chown","dac\_override","dac\_read\_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux\_immutable","net\_bind\_service","net\_broadcast","net\_admin","net\_raw","ipc\_lock","ipc\_owner","sys\_module","sys\_rawio","sys\_chroot","sys\_ptrace","sys\_pacct","sys\_admin","sys\_boot","sys\_nice","sys\_resource","sys\_time","sys\_tty\_config","mknod","lease","audit\_write","audit\_control","setfcap","mac\_override","mac\_admin","syslog","wake\_alarm","block\_suspend"],"ambient":null}, "cwd": "/root", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 9954, "ppid": 1821, "seccomp": {"mode":"disabled"}, "start\_time": "2019-09-01T06:36:50.960-0400"}}}  
2019-09-01T06:36:54.939-0400 INFO instance/beat.go:292 Setup Beat: metricbeat; Version: 7.3.1  
2019-09-01T06:36:54.939-0400 DEBUG [beat] instance/beat.go:318 Initializing output plugins  
2019-09-01T06:36:54.939-0400 INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'metricbeat-7.3.1' as ILM is enabled.  
2019-09-01T06:36:54.939-0400 INFO elasticsearch/client.go:170 Elasticsearch url: [http://localhost:9200](http://localhost:9200)  
2019-09-01T06:36:54.940-0400 DEBUG [publisher] pipeline/consumer.go:137 start pipeline event consumer  
2019-09-01T06:36:54.940-0400 INFO [publisher] pipeline/module.go:97 Beat name: elk.tongyao.com.au  
2019-09-01T06:36:54.940-0400 DEBUG [modules] beater/metricbeat.go:121 Available modules and metricsets: Register [ModuleFactory:[aws, beat, docker, elasticsearch, kibana, logstash, mongodb, mssql, mysql, oracle, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, aws/cloudwatch, aws/ec2, aws/rds, aws/s3\_daily\_storage, aws/s3\_request, aws/sqs, beat/state, beat/stats, ceph/cluster\_disk, ceph/cluster\_health, ceph/cluster\_status, ceph/monitor\_health, ceph/osd\_df, ceph/osd\_tree, ceph/pool\_disk, consul/agent, coredns/stats, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster\_stats, elasticsearch/index, elasticsearch/index\_recovery, elasticsearch/index\_summary, elasticsearch/ml\_job, elasticsearch/node, elasticsearch/node\_stats, elasticsearch/pending\_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state\_container, kubernetes/state\_deployment, kubernetes/state\_node, kubernetes/state\_pod, kubernetes/state\_replicaset, kubernetes/state\_statefulset, kubernetes/system, kubernetes/volume, kvm/dommemstat, logstash/node, logstash/node\_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, mssql/performance, mssql/transaction\_log, munin/node, mysql/galera\_status, mysql/status, nats/connections, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, oracle/tablespace, php\_fpm/pool, php\_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/process\_summary, system/raid, system/socket, system/socket\_summary, system/uptime, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[cockroachdb/status]]  
2019-09-01T06:36:54.941-0400 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s  
2019-09-01T06:36:54.942-0400 INFO instance/beat.go:421 metricbeat start running.  
2019-09-01T06:36:54.942-0400 INFO cfgfile/reload.go:171 Config reloader started  
2019-09-01T06:37:04.943-0400 DEBUG [cfgfile] cfgfile/reload.go:197 Scan for new config files

---

<div class="post-metadata">

**Author:** ![Nee\_Defeng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nee_defeng/32/45771_2.png) [@Nee\_Defeng](https://discuss.elastic.co/u/Nee_Defeng)\
**Post date:** [September 2, 2019, 7:21am UTC](https://discuss.elastic.co/t/aws-module/197618/3 "2019-09-02T07:21:25Z")

</div>

The issue was caused by IAM user permission. Once the policy was created and assigned to the IAM user, the module started to collect events from AWS.

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [September 3, 2019, 1:24pm UTC](https://discuss.elastic.co/t/aws-module/197618/4 "2019-09-03T13:24:48Z")

</div>

Glad to see that you made it work.

Thanks for posting the solution 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 1, 2019, 1:34pm UTC](https://discuss.elastic.co/t/aws-module/197618/5 "2019-10-01T13:34:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.