# Issues configuring AWS Module for MetricBeat **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [June 2, 2020, 2:24pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355 "2020-06-02T14:24:24Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [June 2, 2020, 2:24pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/1 "2020-06-02T14:24:24Z")
Hello. I'm attempting to set up the AWS module for the first time with MetricBeat. I've got the module enabled, got past the authentication errors I was seeing (created an AWS user with full "ReadOnlyPolicy"), and updated the aws.yml file. I am seeing a little bit of information, which is telling me that my account authorization is working. However, the only 2 services I'm seeing statistics for is EC2 and S3. I'm using the Kibana dashboards I loaded from Elastic for AWS. Here are the issues I'm seeing: EC2 - only seeing 5 instances - choices seem random EBS - no statistics at all RDS - no statistics at all Usage - seeing minor stats for EC2 and Cloudwatch, but dashboard is saying: Could not locate that index-pattern-field (id: aws.usage.metrics.ResourceCount.sum) Could not locate that index-pattern-field (id: aws.usage.metrics.CallCount.sum) So, I'm definitely connected, but I'm getting very inconsistent data. Here is what my YML looks like: ```auto - module: aws period: 5m metricsets: - cloudwatch metrics: - namespace: "*" #- namespace: AWS/EC2 #name: ["CPUUtilization", "DiskWriteOps"] # tags.resource_type_filter: ec2:instance #- namespace: AWS/EBS #- namespace: AWS/ELB #- namespace: AWS/Billing #- namespace: AWS/DX #- namespace: AWS/ElasticBeanstalk #- namespace: AWS/Lambda #- namespace: AWS/Redshift #- namespace: AWS/RDS #- namespace: AWS/Route53 #- namespace: AWS/S3 #- namespace: AWS/VPN access_key_id: 'key' secret_access_key: key’ - module: aws period: 5m metricsets: - dynamodb - ebs - ec2 - lambda - rds - sns - sqs - vpn - usage - elb access_key_id: ‘key’ secret_access_key: ‘key’ - module: aws period: 12h metricsets: - billing regions: - us-east-1 access_key_id: ‘key’ secret_access_key: ‘key’ - module: aws period: 24h metricsets: - s3_daily_storage - s3_request access_key_id: ‘key’ secret_access_key: ‘key’ ``` ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [June 2, 2020, 3:58pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/2 "2020-06-02T15:58:12Z")
Have you run `./metribeat setup` to deploy assets into Kibana and Elasticsearch? ---
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [June 2, 2020, 4:17pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/3 "2020-06-02T16:17:18Z")
@kvch Yes, I ran it. I had first set up metricbeat to gather some CPU statistics and such, so I believe I ran it at that time. I did a quick "history" check on Linux and I did run this a couple of times. I have since moved on to try and setup AWS and are running into these issues. ---
**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\ **Post date:** [June 3, 2020, 6:08pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/4 "2020-06-03T18:08:53Z")
Hi! For only seeing 5 EC2 instances, do you mean from the EC2 dashboard? That's because in the dashboard, for better view, we have a limit on how many instances to show (top 5). You can change this limit under each visualization. For config like this: ```auto - module: aws period: 5m metricsets: - cloudwatch metrics: - namespace: "*" ``` You will be collecting all metrics from CloudWatch, which _might_ introduce a large amount of AWS API cost. Just want to make sure you are aware of this 😄 Regarding to no statistics for EBS/RDS, do you see metrics from EBS/RDS in Kibana Discover? ---
**Author:** ![ozonshak](https://avatars.discourse-cdn.com/v4/letter/o/eb9ed0/32.png) [@ozonshak](https://discuss.elastic.co/u/ozonshak)\ **Post date:** [June 4, 2020, 1:30pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/5 "2020-06-04T13:30:02Z")
Thanks @Kaiyan_Sheng. Ok, so I found the visualization limit for EC2 and changed that from 5 to 20. So, I'm good there. For RDS, I noticed in the visualization that the default index pattern was something other than metricbeat for some reason. So, I changed that to MetricBeat: ![wrong_index](https://us1.discourse-cdn.com/elastic/original/3X/0/2/027f98a7457c04a4f54b8dbd36a8edcafa60400a.jpeg) Next, I picked one of the visualizations - just chose the "database connections" one. Here I can see the correct index now, but there was no field selected. So, I chose aws.rds.database\_connections. That seems to be populating that visualiaztion with data now, however, this is going to be a bit painful to try and choose the appropriate fields for all the visualizations. Any idea why these are not set and what they should be for the various visualizations? ![rds_field](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7d77877fa66870d6b40c6781994190aa439d119b.jpeg) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 2, 2020, 1:30pm UTC](https://discuss.elastic.co/t/issues-configuring-aws-module-for-metricbeat/235355/6 "2020-07-02T13:30:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.