# AWS metricbeat unable to load data in kibana

**URL:** <https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [September 6, 2021, 7:40am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416 "2021-09-06T07:40:37Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [September 6, 2021, 7:40am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/1 "2021-09-06T07:40:37Z")

</div>

Hi ,

I am using metricbeat 7.14. IAM policy given for AWS billing and AWS.yml config file is as follows as stated in

> [AWS billing metricset | Metricbeat Reference [7.16] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/7.x/metricbeat-metricset-aws-billing.html)

Step I followed to start aws and flow aws billing data to kibana are as follows.

S1\>Make changes in metricbeat.yml (changed cloud id,user/password)  
S2\>Enable AWS  
S3\>Make changes in AWS.yml  
S4\>Start metricbeat, but data didn't flow.

When I checked the status of metricbeat, i am getting failed message.

```auto
sudo service metricbeat status
● metricbeat.service - Metricbeat is a lightweight shipper for metrics.
   Loaded: loaded (/usr/lib/systemd/system/metricbeat.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Mon 2021-09-06 07:33:16 UTC; 23s ago
     Docs: https://www.elastic.co/beats/metricbeat
  Process: 24312 ExecStart=/usr/share/metricbeat/bin/metricbeat --environment systemd $BEAT_LOG_OPTS $BEAT_CONFIG_OPTS $BEAT_PATH_OPTS (code=exited, status=1/FAILURE)
 Main PID: 24312 (code=exited, status=1/FAILURE)

Sep 06 07:33:16 dev systemd[1]: Unit metricbeat.service entered failed state.
Sep 06 07:33:16 dev systemd[1]: metricbeat.service failed.
Sep 06 07:33:16 dev systemd[1]: metricbeat.service holdoff time over, scheduling restart.
Sep 06 07:33:16 dev systemd[1]: Stopped Metricbeat is a lightweight shipper for metrics..
Sep 06 07:33:16 dev systemd[1]: start request repeated too quickly for metricbeat.service
Sep 06 07:33:16 dev systemd[1]: Failed to start Metricbeat is a lightweight shipper for metrics..
Sep 06 07:33:16 dev systemd[1]: Unit metricbeat.service entered failed state.
Sep 06 07:33:16 dev systemd[1]: metricbeat.service failed.

```

As metricbeat is not starting then I disable aws module and try starting metricbeat again.Then Metrcibeat gets started.  
Then I enable aws module but still aws data is not flowing to the kibana,

Pls let me know if i am making mistake in running the step or in aws.yml file.

```auto
sudo service metricbeat status
● metricbeat.service - Metricbeat is a lightweight shipper for metrics.
   Loaded: loaded (/usr/lib/systemd/system/metricbeat.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2021-09-06 07:36:47 UTC; 7s ago
     Docs: https://www.elastic.co/beats/metricbeat
 Main PID: 27599 (metricbeat)
    Tasks: 10
   Memory: 55.5M
   CGroup: /system.slice/metricbeat.service
           └─27599 /usr/share/metricbeat/bin/metricbeat --environment systemd -c /etc/metricbeat/metricbeat.yml --path.home /usr/share/metricbeat --path.config /etc/...

Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.773Z INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.807Z INFO [index-management.ilm] ilm/std.go:160 ILM policy metricbea...s already.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.807Z INFO [index-management] idxmgmt/std.go:401 Set setup.template.n...s enabled.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.807Z INFO [index-management] idxmgmt/std.go:406 Set setup.template.p...s enabled.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.807Z INFO [index-management] idxmgmt/std.go:440 Set settings.index.l...s enabled.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.807Z INFO [index-management] idxmgmt/std.go:444 Set settings.index.l...s enabled.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.843Z INFO template/load.go:111 Template "metricbeat-7.14.0" already exists ...erwritten.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.843Z INFO [index-management] idxmgmt/std.go:297 Loaded index template.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.883Z INFO [index-management.ilm] ilm/std.go:121 Index Alias metricbe...s already.
Sep 06 07:36:48 dev metricbeat[27599]: 2021-09-06T07:36:48.884Z INFO [publisher_pipeline_output] pipeline/output.go:151 Connect...stablished
Hint: Some lines were ellipsized, use -l to show in full.

```

Aws.yml

```auto
- module: aws
   period: 60s
  metricsets:
    - billing
  cost_explorer_config:
    group_by_dimension_keys:
      - "AZ"
      - "INSTANCE_TYPE"
      - "SERVICE"
      - "LINKED_ACCOUNT"
    group_by_tag_keys:
      - "aws:createdBy"
  access_key_id: '${AWS_ACCESS_KEY_ID:"<>"}'
  secret_access_key: '${AWS_SECRET_ACCESS_KEY:"< >"}'

```

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [September 8, 2021, 6:53am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/2 "2021-09-08T06:53:46Z")

</div>

Hi @Divyank_Mahalle !

Do you see anything related in Metricbeat's logs? If it is an issue with your configuration you should be able to see the error in Metricbear's logs.

C.

---

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [September 9, 2021, 10:02am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/3 "2021-09-09T10:02:04Z")

</div>

Hi Chris,  
I am not able to seeing any error in metricbeat logs.

I checked with IAM polciy, its applies correctly as per 7.14.  
Few days back I am able to aws logs to kibana with ES version 7.8,post upgradation aws logs are not loading but system metrics data is loading successfully in metricbeat-\* index in old and new ES 7.8 and 7.14 version.  
I am not getting where I am doing mistake.All config is same just 1 IAM policy organizationaccount is added for 7.14 version.

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [September 9, 2021, 10:14am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/4 "2021-09-09T10:14:35Z")

</div>

@Kaiyan_Sheng would you mild checking if there is anything obvious here?

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 9, 2021, 11:46am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/5 "2021-09-09T11:46:43Z")

</div>

Can you post the Metricbeat logs. If its not showing much can you try to manually start Metricbeat from the CLI using the `-e` flag to run in debug mode. That will output a lot more data that we can use to diagnosis the issue.

---

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [September 10, 2021, 10:23am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/7 "2021-09-10T10:23:56Z")

</div>

Current Logs-Now no errors and aws details are also getting loaded in logs but still aws data is not flowing.  
Pls let me know any changes required in my aws.yml config or any logs required.

> DEBUG [cfgfile] cfgfile/reload.go:213 Number of module configs found: 0

```auto
sudo metricbeat -e -d "*"

2021-09-10T08:42:04.167-0400 INFO instance/beat.go:665 Home path: [/usr/share/metricbeat] Config path: [/etc/metricbeat] Data path: [/var/lib/metricbeat] Logs path: [/var/log/metricbeat]
2021-09-10T08:42:04.167-0400 DEBUG [beat] instance/beat.go:723 Beat metadata path: /var/lib/metricbeat/meta.json
2021-09-10T08:42:04.167-0400 INFO instance/beat.go:673 Beat ID: 0234d09a-49fa-471a-8193-62ca98cbdd15
2021-09-10T08:42:04.171-0400 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:165 add_cloud_metadata: received disposition for aws after 1.757165ms. result=[provider:aws, error=<nil>, metadata={"cloud":{"account":{"id":"<>"},"availability_zone":"us-east-1a","image":{"id":"<>"},"instance":{"id":"<>"},"machine":{"type":"<>"},"provider":"aws","region":"us-east-1","service":{"name":"EC2"}}}]
2021-09-10T08:42:04.171-0400 DEBUG [add_cloud_metadata] add_cloud_metadata/providers.go:131 add_cloud_metadata: fetchMetadata ran for 1.844447ms
2021-09-10T08:42:04.171-0400 INFO [add_cloud_metadata] add_cloud_metadata/add_cloud_metadata.go:105 add_cloud_metadata: hosting provider type detected as aws, metadata={"cloud":{"account":{"id":"<>"},"availability_zone":"us-east-1a","image":{"id":"<>"},"instance":{"id":"i-030b87c242e566df2"},"machine":{"type":"t3.large"},"provider":"aws","region":"us-east-1","service":{"name":"EC2"}}}
2021-09-10T08:42:04.171-0400 INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
2021-09-10T08:42:04.172-0400 INFO [beat] instance/beat.go:1014 Beat info {"system_info": {"beat": {"path": {"config": "/etc/metricbeat", "data": "/var/lib/metricbeat", "home": "/usr/share/metricbeat", "logs": "/var/log/metricbeat"}, "type": "metricbeat", "uuid": "0234d09a-49fa-471a-8193-62ca98cbdd15"}}}
2021-09-10T08:42:04.172-0400 INFO [beat] instance/beat.go:1023 Build info {"system_info": {"build": {"commit": "e127fc31fc6c00fdf8649808f9421d8f8c28b5db", "libbeat": "7.14.0", "time": "2021-07-29T21:09:05.000Z", "version": "7.14.0"}}}
2021-09-10T08:42:04.172-0400 INFO [beat] instance/beat.go:1026 Go runtime info {"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":2,"version":"go1.16.6"}}}
2021-09-10T08:42:04.172-0400 INFO [beat] instance/beat.go:1030 Host info {"system_info": {"host": {"architecture":"x86_64","boot_time":"2021-02-22T08:56:08-05:00","containerized":false,"name":"dev-0","ip":["127.0.0.1/8","::1/128","172.30.0.63/24","fe80::c8a:d3ff:fe97:b675/64"],"kernel_version":"5.4.0-1037-aws","mac":["0e:8a:d3:97:b6:75"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"18.04.5 LTS (Bionic Beaver)","major":18,"minor":4,"patch":5,"codename":"bionic"},"timezone":"EDT","timezone_offset_sec":-14400,"id":"ec211df81d25fc20b5021dac6516d70c"}}}
2021-09-10T08:42:04.173-0400 INFO instance/beat.go:309 Setup Beat: metricbeat; Version: 7.14.0
2021-09-10T08:42:04.173-0400 DEBUG [beat] instance/beat.go:335 Initializing output plugins
2021-09-10T08:42:04.173-0400 INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'metricbeat-7.14.0' as ILM is enabled.
2021-09-10T08:42:04.173-0400 INFO [esclientleg] eslegclient/connection.go:100 elasticsearch url: <>
2021-09-10T08:42:04.173-0400 DEBUG [publisher] pipeline/consumer.go:148 start pipeline event consumer
2021-09-10T08:42:04.173-0400 INFO [publisher] pipeline/module.go:113 Beat name: dev-0
2021-09-10T08:42:04.220-0400 INFO instance/beat.go:473 metricbeat start running.
2021-09-10T08:42:04.220-0400 DEBUG [cfgfile] cfgfile/reload.go:132 Checking module configs from: /etc/metricbeat/modules.d/*.yml
2021-09-10T08:42:04.221-0400 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-09-10T08:42:04.221-0400 DEBUG [cfgfile] cfgfile/reload.go:146 Number of module configs found: 0
2021-09-10T08:42:04.220-0400 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2021-09-10T08:42:04.221-0400 INFO cfgfile/reload.go:164 Config reloader started
2021-09-10T08:42:04.221-0400 DEBUG [cfgfile] cfgfile/reload.go:194 Scan for new config files
2021-09-10T08:42:04.221-0400 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-09-10T08:42:04.221-0400 DEBUG [cfgfile] cfgfile/reload.go:213 Number of module configs found: 0
2021-09-10T08:42:04.221-0400 DEBUG [reload] cfgfile/list.go:63 Starting reload procedure, current runners: 0
2021-09-10T08:42:04.221-0400 DEBUG [reload] cfgfile/list.go:81 Start list: 0, Stop list: 0
2021-09-10T08:42:04.221-0400 INFO cfgfile/reload.go:224 Loading of config files completed.

```

aws.yml config-

```auto
metricbeat.modules:
- module: aws
  period: 60s
  metricsets:
     - billing
  access_key_id: '${AWS_ACCESS_KEY_ID:"<>"}'
  secret_access_key: '${AWS_SECRET_ACCESS_KEY:"<>"}'

```

While I changed aws.yml confg its throwing error.

> instance/beat.go:989 Exiting: 1 error: error creating aws metricset: failed DescribeRegions: UnauthorizedOperation: You are not authorized to perform this operation.

```auto
2021-09-10T10:40:13.350-0400 INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2021-09-10T10:40:13.350-0400 INFO instance/beat.go:473 metricbeat start running.
2021-09-10T10:40:13.350-0400 DEBUG [cfgfile] cfgfile/reload.go:132 Checking module configs from: /etc/metricbeat/modules.d/*.yml
2021-09-10T10:40:13.351-0400 DEBUG [cfgfile] cfgfile/cfgfile.go:193 Load config from file: /etc/metricbeat/modules.d/aws.yml
2021-09-10T10:40:13.351-0400 DEBUG [cfgfile] cfgfile/reload.go:146 Number of module configs found: 1
2021-09-10T10:40:13.351-0400 DEBUG [getAccessKeys] aws/credentials.go:69 Using access keys for AWS credential
2021-09-10T10:40:13.351-0400 DEBUG [aws.billing] aws/aws.go:89 aws config endpoint =
2021-09-10T10:40:13.351-0400 DEBUG [aws.billing] aws/aws.go:99 Metricset level config for period: 12h0m0s
2021-09-10T10:40:13.351-0400 DEBUG [aws.billing] aws/aws.go:100 Metricset level config for tags filter: []
2021-09-10T10:40:13.351-0400 WARN [aws.billing] aws/aws.go:101 extra charges on AWS API requests will be generated by this metricset
2021-09-10T10:40:13.415-0400 DEBUG [aws.billing] aws/aws.go:131 AWS Credentials belong to account ID: <>
2021-09-10T10:40:13.445-0400 DEBUG [aws.billing] aws/aws.go:193 AWS Credentials belong to account name:
2021-09-10T10:42:46.051-0400 ERROR instance/beat.go:989 Exiting: 1 error: error creating aws metricset: failed DescribeRegions: UnauthorizedOperation: You are not authorized to perform this operation.

```

Aws.yml config-(I removed first line metricbeat.modules in it.

```auto
- module: aws
  period: 60s
  metricsets:
    - billing
  access_key_id: "< >"
  secret_access_key: "< >"

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 11, 2021, 12:16am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/8 "2021-09-11T00:16:17Z")

</div>

> [@Divyank\_Mahalle](#):
>
> ```auto
> 2021-09-10T10:42:46.051-0400 ERROR instance/beat.go:989 Exiting: 1 error: error creating aws metricset: failed DescribeRegions: UnauthorizedOperation: You are not authorized to perform this operation.
> 
> ```

Seems like you didn't give ur user/keys the correct IAM roles/permissions

---

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [September 13, 2021, 5:53am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/9 "2021-09-13T05:53:10Z")

</div>

Hi

Thanks for your valuable inputs, billing data successfully flowing through kibana after adding "ec2:DescribeRegions" policy.

But in 7.14 metrcibeat official documentation this policy is not present.

My working IAM policy looks like this-

```auto
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "organizations:ListAccounts",
                "tag:GetResources",
                "cloudwatch:GetMetricData",
                "iam:ListAccountAliases",
                "ce:GetCostAndUsage",
                "sts:GetCallerIdentity",
                "cloudwatch:ListMetrics",
                "ec2:DescribeRegions"
            ],
            "Resource": "*"
        }
    ]
}

```

---

<div class="post-metadata">

**Author:** ![legoguy1000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/legoguy1000/32/54301_2.png) [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)\
**Post date:** [September 13, 2021, 11:06am UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/10 "2021-09-13T11:06:58Z")

</div>

This is what I see in the docs., [AWS usage metricset | Metricbeat Reference [7.14] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-aws-usage.html#_aws_permissions_15). if you think it's missing permissions, feel free to open a GitHub issue.

---

<div class="post-metadata">

**Author:** ![Divyank\_Mahalle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divyank_mahalle/32/91240_2.png) [@Divyank\_Mahalle](https://discuss.elastic.co/u/Divyank_Mahalle)\
**Post date:** [September 13, 2021, 12:19pm UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/11 "2021-09-13T12:19:00Z")

</div>

Ah yes, in usage metricset "ec2:DescribeRegions" is present.

As I was checking for billing metricset for aws billing data, "ec2:DescribeRegions" is not present.

> **[AWS billing metricset | Metricbeat Reference \[7.14\] | Elastic](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-aws-billing.html)**

But anyways, Thanks once again.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 11, 2021, 2:19pm UTC](https://discuss.elastic.co/t/aws-metricbeat-unable-to-load-data-in-kibana/283416/12 "2021-10-11T14:19:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.