I am using the Metricbeat AWS module while starting Metricbeat, I am facing the following error. I went through Git issues but seems it's fixed in the version I am using. I am pasting my aws.yml as well for reference.
Error
2021-03-12T07:36:48.222Z INFO instance/beat.go:640 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2021-03-12T07:36:48.223Z INFO instance/beat.go:648 Beat ID: 525ded62-ce68-4a71-85d4-3e09f56afc20
2021-03-12T07:36:48.227Z INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:93 add_cloud_metadata: hosting provider type detected as aws, metadata={"account":{"id":"917006514933"},"availability_zone":"ap-south-1a","image":{"id":"ami-042b34111b1289ccd"},"instance":{"id":"i-0f790ff5d45cffd48"},"machine":{"type":"t2.micro"},"provider":"aws","region":"ap-south-1"}
2021-03-12T07:36:48.227Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
2021-03-12T07:36:48.228Z INFO [beat] instance/beat.go:976 Beat info {"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "525ded62-ce68-4a71-85d4-3e09f56afc20"}}}
2021-03-12T07:36:48.228Z INFO [beat] instance/beat.go:985 Build info {"system_info": {"build": {"commit": "ad823eca4cc74439d1a44351c596c12ab51054f5", "libbeat": "7.9.1", "time": "2020-09-01T20:03:55.000Z", "version": "7.9.1"}}}
2021-03-12T07:36:48.228Z INFO [beat] instance/beat.go:988 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":1,"version":"go1.14.7"}}}
2021-03-12T07:36:48.229Z INFO [beat] instance/beat.go:992 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-03-12T05:12:51Z","containerized":false,"name":"ip-172-31-33-55","ip":["127.0.0.1/8","::1/128","172.31.33.55/20","fe80::a:6aff:fea2:7078/64"],"kernel_version":"4.4.0-1122-aws","mac":["02:0a:6a:a2:70:78"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.7 LTS (Xenial Xerus)","major":16,"minor":4,"patch":7,"codename":"xenial"},"timezone":"UTC","timezone_offset_sec":0,"id":"1ce62cb7754341cbbec8a0bf54a731bb"}}}
2021-03-12T07:36:48.229Z INFO [beat] instance/beat.go:1021 Process info {"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/root", "exe": "/usr/share/metricbeat/bin/metricbeat", "name": "metricbeat", "pid": 2409, "ppid": 2408, "seccomp": {"mode":"filter"}, "start_time": "2021-03-12T07:36:47.379Z"}}}
2021-03-12T07:36:48.230Z INFO instance/beat.go:299 Setup Beat: metricbeat; Version: 7.9.1
2021-03-12T07:36:48.230Z INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.9.1' as ILM is enabled.
2021-03-12T07:36:48.232Z INFO eslegclient/connection.go:99 elasticsearch url: https://65.0.82.214:9200
2021-03-12T07:36:48.232Z WARN [tls] tlscommon/tls_config.go:83 SSL/TLS verifications disabled.
2021-03-12T07:36:48.233Z INFO [publisher] pipeline/module.go:113 Beat name: ip-172-31-33-55
2021-03-12T07:36:48.247Z INFO instance/beat.go:450 metricbeat start running.
2021-03-12T07:36:48.251Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2021-03-12T07:36:49.080Z INFO [monitoring] log/log.go:153 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":10,"time":{"ms":16}},"total":{"ticks":150,"time":{"ms":160},"value":0},"user":{"ticks":140,"time":{"ms":144}}},"handles":{"limit":{"hard":1048576,"soft":1024},"open":9},"info":{"ephemeral_id":"b649aa5e-100c-4516-b264-f91e491ce021","uptime":{"ms":909}},"memstats":{"gc_next":21903952,"memory_alloc":16880272,"memory_total":39375568,"rss":82509824},"runtime":{"goroutines":19}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0}}},"system":{"cpu":{"cores":1},"load":{"1":0.08,"15":0.01,"5":0.02,"norm":{"1":0.08,"15":0.01,"5":0.02}}}}}}
2021-03-12T07:36:49.081Z INFO [monitoring] log/log.go:154 Uptime: 910.492701ms
2021-03-12T07:36:49.081Z INFO [monitoring] log/log.go:131 Stopping metrics logging.
2021-03-12T07:36:49.081Z INFO runtime/panic.go:975 metricbeat stopped.
2021-03-12T07:36:49.082Z FATAL [metricbeat] instance/beat.go:164 Failed due to panic. {"panic": "runtime error: index out of range [0] with length 0", "stack": "github.com/elastic/beats/v7/libbeat/cmd/instance.Run.func1.1\n\t/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:165\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:969\nruntime.goPanicIndex\n\t/usr/local/go/src/runtime/panic.go:88\ngithub.com/elastic/beats/v7/x-pack/metricbeat/module/aws.NewMetricSet\n\t/go/src/github.com/elastic/beats/x-pack/metricbeat/module/aws/aws.go:122\ngithub.com/elastic/beats/v7/x-pack/metricbeat/module/aws/cloudwatch.New\n\t/go/src/github.com/elastic/beats/x-pack/metricbeat/module/aws/cloudwatch/cloudwatch.go:113\ngithub.com/elastic/beats/v7/metricbeat/mb.(*LightMetricSet).Registration.func1\n\t/go/src/github.com/elastic/beats/metricbeat/mb/lightmetricset.go:96\ngithub.com/elastic/beats/v7/metricbeat/mb.initMetricSets\n\t/go/src/github.com/elastic/beats/metricbeat/mb/builders.go:139\ngithub.com/elastic/beats/v7/metricbeat/mb.NewModule\n\t/go/src/github.com/elastic/beats/metricbeat/mb/builders.go:65\ngithub.com/elastic/beats/v7/metricbeat/mb/module.NewWrapper\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:83\ngithub.com/elastic/beats/v7/metricbeat/mb/module.(*Factory).CheckConfig\n\t/go/src/github.com/elastic/beats/metricbeat/mb/module/factory.go:78\ngithub.com/elastic/beats/v7/libbeat/cfgfile.(*Reloader).Check\n\t/go/src/github.com/elastic/beats/libbeat/cfgfile/reload.go:155\ngithub.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run\n\t/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:234\ngithub.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).launch\n\t/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:456\ngithub.com/elastic/beats/v7/libbeat/cmd/instance.Run.func1\n\t/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:189\ngithub.com/elastic/beats/v7/libbeat/cmd/instance.Run\n\t/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:190\ngithub.com/elastic/beats/v7/libbeat/cmd.genRunCmd.func1\n\t/go/src/github.com/elastic/beats/libbeat/cmd/run.go:36\ngithub.com/spf13/cobra.(*Command).execute\n\t/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:766\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\t/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852\ngithub.com/spf13/cobra.(*Command).Execute\n\t/go/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:800\nmain.main\n\t/go/src/github.com/elastic/beats/x-pack/metricbeat/main.go:21\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203"}
aws.yml file
# Module: aws
# Docs: AWS module | Metricbeat Reference [7.9] | Elastic
- module: aws
period: 1m
access_key_id: "access_key"
secret_access_key: "secret_key"
metricsets:
- elb
- natgateway
- rds
- transitgateway
- usage
- vpn
- module: aws
period: 5m
access_key_id: "access_key"
secret_access_key: "secret_key"
metricsets:
- cloudwatch
metrics:
- namespace: AWS/EC2
#name: ["CPUUtilization", "DiskWriteOps"]
resource_type: ec2:instance
#dimensions:
# - name: InstanceId
# value: i-0686946e22cf9494a
#statistic: ["Average", "Maximum"]
- module: aws
period: 5m
access_key_id: "access_key"
secret_access_key: "secret_key"
metricsets:
- dynamodb
- ebs
- ec2
- lambda
- rds
- sns
- sqs
- module: aws
period: 12h
access_key_id: "access_key"
secret_access_key: "secret_key"
metricsets:
- billing
regions:
- ap-south-1
- module: aws
period: 24h
access_key_id: "access_key"
secret_access_key: "secret_key"
metricsets:
- s3_daily_storage
- s3_request
The permissions AWS keys having is as following
Thanks for posting here!! Could you send me the github issue link please? TIA!!