Aws module

Elastic Stack Beats
1

There was no error in the debug log, and there was no events pulled from AWS (in metricbeat-* index there was no message).

Please help, thanks in advance!

Here is the config:

# Module: aws
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.3/metricbeat-module-aws.html

- module: aws
  period: 300s
  metricsets:
    - ec2
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
- module: aws
  period: 300s
  metricsets:
    - sqs
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
  #regions:
  #  - us-west-1
- module: aws
  period: 86400s
  metricsets:
    - s3_request
    - s3_daily_storage
  access_key_id: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  secret_access_key: 'XXXXXXXXXXXXXXXXXXXXXXXX'
  default_region: 'us-west-2'
  #regions:
  #  - us-west-1

Here is the debug log:

[root@elk ~]# metricbeat -e -d "*"
2019-09-01T06:36:51.932-0400 INFO instance/beat.go:606 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2019-09-01T06:36:51.932-0400 DEBUG [beat] instance/beat.go:658 Beat metadata path: /var/lib/metricbeat/meta.json
2019-09-01T06:36:51.932-0400 INFO instance/beat.go:614 Beat ID: cb4997de-b548-4ea4-9a48-9ab8fbb9dabe
2019-09-01T06:36:51.935-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:164 add_cloud_metadata: starting to fetch metadata, timeout=3s
2019-09-01T06:36:51.948-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for gcp after 12.560124ms. result=[provider:gcp, error=failed requesting gcp metadata: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json: dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]
2019-09-01T06:36:51.948-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for openstack after 12.731171ms. result=[provider:openstack, error=failed requesting openstack metadata: Get http://169.254.169.254/2009-04-04/meta-data/instance-id: dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]
2019-09-01T06:36:51.948-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for aws after 12.812023ms. result=[provider:aws, error=failed requesting aws metadata: Get http://169.254.169.254/2014-02-25/dynamic/instance-identity/document: dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]
2019-09-01T06:36:51.948-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for digitalocean after 12.964448ms. result=[provider:digitalocean, error=failed requesting digitalocean metadata: Get http://169.254.169.254/metadata/v1.json: dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]
2019-09-01T06:36:51.950-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for az after 14.051429ms. result=[provider:az, error=failed requesting az metadata: Get http://169.254.169.254/metadata/instance/compute?api-version=2017-04-02: dial tcp 169.254.169.254:80: connect: connection refused, metadata={}]
2019-09-01T06:36:52.558-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for qcloud after 622.875441ms. result=[provider:qcloud, error=failed requesting qcloud metadata: Get http://metadata.tencentyun.com/meta-data/instance-id: dial tcp: lookup metadata.tencentyun.com on 192.168.245.2:53: no such host, metadata={}]
2019-09-01T06:36:54.937-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:196 add_cloud_metadata: received disposition for ecs after 3.001561093s. result=[provider:ecs, error=failed requesting ecs metadata: Get http://100.100.100.200/latest/meta-data/region-id: net/http: request canceled (Client.Timeout exceeded while awaiting headers), metadata={}]
2019-09-01T06:36:54.937-0400 DEBUG [filters] add_cloud_metadata/add_cloud_metadata.go:167 add_cloud_metadata: fetchMetadata ran for 3.001629667s
2019-09-01T06:36:54.937-0400 INFO add_cloud_metadata/add_cloud_metadata.go:347 add_cloud_metadata: hosting provider type not detected.
2019-09-01T06:36:54.937-0400 DEBUG [processors] processors/processor.go:93 Generated new processors: add_host_metadata=[netinfo.enabled=[false], cache.ttl=[5m0s]], add_cloud_metadata=null
2019-09-01T06:36:54.937-0400 INFO [seccomp] seccomp/seccomp.go:101 Syscall filter could not be installed because the kernel does not support seccomp

2

2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:902 Beat info {"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "cb4997de-b548-4ea4-9a48-9ab8fbb9dabe"}}}
2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:911 Build info {"system_info": {"build": {"commit": "a4be71b90ce3e3b8213b616adfcd9e455513da45", "libbeat": "7.3.1", "time": "2019-08-19T19:33:03.000Z", "version": "7.3.1"}}}
2019-09-01T06:36:54.937-0400 INFO [beat] instance/beat.go:914 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":8,"version":"go1.12.4"}}}
2019-09-01T06:36:54.939-0400 INFO [beat] instance/beat.go:918 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-09-01T04:54:26-04:00","containerized":false,"name":"elk.tongyao.com.au","ip":["127.0.0.1/8","::1/128","192.168.245.129/24","fe80::6e39:8fcb:710:ee1a/64"],"kernel_version":"3.10.0-693.el7.x86_64","mac":["00:0c:29:a9:9b:c4"],"os":{"family":"redhat","platform":"rhel","name":"Red Hat Enterprise Linux Server","version":"7.4 (Maipo)","major":7,"minor":4,"patch":0,"codename":"Maipo"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"a43e1577a5804785a131ba4d1ef34ea3"}}}
2019-09-01T06:36:54.939-0400 INFO [beat] instance/beat.go:947 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend"],"ambient":null}, "cwd": "/root", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 9954, "ppid": 1821, "seccomp": {"mode":"disabled"}, "start_time": "2019-09-01T06:36:50.960-0400"}}}
2019-09-01T06:36:54.939-0400 INFO instance/beat.go:292 Setup Beat: metricbeat; Version: 7.3.1
2019-09-01T06:36:54.939-0400 DEBUG [beat] instance/beat.go:318 Initializing output plugins
2019-09-01T06:36:54.939-0400 INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'metricbeat-7.3.1' as ILM is enabled.
2019-09-01T06:36:54.939-0400 INFO elasticsearch/client.go:170 Elasticsearch url: http://localhost:9200
2019-09-01T06:36:54.940-0400 DEBUG [publisher] pipeline/consumer.go:137 start pipeline event consumer
2019-09-01T06:36:54.940-0400 INFO [publisher] pipeline/module.go:97 Beat name: elk.tongyao.com.au
2019-09-01T06:36:54.940-0400 DEBUG [modules] beater/metricbeat.go:121 Available modules and metricsets: Register [ModuleFactory:[aws, beat, docker, elasticsearch, kibana, logstash, mongodb, mssql, mysql, oracle, postgresql, system, uwsgi], MetricSetFactory:[aerospike/namespace, apache/status, aws/cloudwatch, aws/ec2, aws/rds, aws/s3_daily_storage, aws/s3_request, aws/sqs, beat/state, beat/stats, ceph/cluster_disk, ceph/cluster_health, ceph/cluster_status, ceph/monitor_health, ceph/osd_df, ceph/osd_tree, ceph/pool_disk, consul/agent, coredns/stats, couchbase/bucket, couchbase/cluster, couchbase/node, couchdb/server, docker/container, docker/cpu, docker/diskio, docker/event, docker/healthcheck, docker/image, docker/info, docker/memory, docker/network, dropwizard/collector, elasticsearch/ccr, elasticsearch/cluster_stats, elasticsearch/index, elasticsearch/index_recovery, elasticsearch/index_summary, elasticsearch/ml_job, elasticsearch/node, elasticsearch/node_stats, elasticsearch/pending_tasks, elasticsearch/shard, envoyproxy/server, etcd/leader, etcd/metrics, etcd/self, etcd/store, golang/expvar, golang/heap, graphite/server, haproxy/info, haproxy/stat, http/json, http/server, jolokia/jmx, kafka/consumergroup, kafka/partition, kibana/stats, kibana/status, kubernetes/apiserver, kubernetes/container, kubernetes/controllermanager, kubernetes/event, kubernetes/node, kubernetes/pod, kubernetes/proxy, kubernetes/scheduler, kubernetes/state_container, kubernetes/state_deployment, kubernetes/state_node, kubernetes/state_pod, kubernetes/state_replicaset, kubernetes/state_statefulset, kubernetes/system, kubernetes/volume, kvm/dommemstat, logstash/node, logstash/node_stats, memcached/stats, mongodb/collstats, mongodb/dbstats, mongodb/metrics, mongodb/replstatus, mongodb/status, mssql/performance, mssql/transaction_log, munin/node, mysql/galera_status, mysql/status, nats/connections, nats/routes, nats/stats, nats/subscriptions, nginx/stubstatus, oracle/tablespace, php_fpm/pool, php_fpm/process, postgresql/activity, postgresql/bgwriter, postgresql/database, postgresql/statement, prometheus/collector, rabbitmq/connection, rabbitmq/exchange, rabbitmq/node, rabbitmq/queue, redis/info, redis/key, redis/keyspace, system/core, system/cpu, system/diskio, system/entropy, system/filesystem, system/fsstat, system/load, system/memory, system/network, system/process, system/process_summary, system/raid, system/socket, system/socket_summary, system/uptime, traefik/health, uwsgi/status, vsphere/datastore, vsphere/host, vsphere/virtualmachine, zookeeper/connection, zookeeper/mntr, zookeeper/server], LightModules:[cockroachdb/status]]
2019-09-01T06:36:54.941-0400 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2019-09-01T06:36:54.942-0400 INFO instance/beat.go:421 metricbeat start running.
2019-09-01T06:36:54.942-0400 INFO cfgfile/reload.go:171 Config reloader started
2019-09-01T06:37:04.943-0400 DEBUG [cfgfile] cfgfile/reload.go:197 Scan for new config files

3

The issue was caused by IAM user permission. Once the policy was created and assigned to the IAM user, the module started to collect events from AWS.

4

Glad to see that you made it work.

Thanks for posting the solution :slight_smile:

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.