Metricbeat AWS module error while using credentials_profile_name

Elastic Stack Beats
1

Hello guys,
I am trying to configure AWS metrics for multiple AWS accounts of mine. I have already configured the profiles using AWS CLI and they are working. But when the same profile name is put in the modules.d/aws.yml it is showing errors.
My configuration and error is below.

Config

modules.d/aws.yml

- module: aws
  period: 1m
  credential_profile_name: elastic-profile
  metricsets:
    - elb
    - usage
- module: aws
  period: 24h
  credential_profile_name: elastic-profile
  metricsets:
    - s3_daily_storage
    - s3_request
- module: aws
  period: 15m
  credential_profile_name: elastic-profile
  metricsets:
    - ebs
    - ec2
    - sns
    - sqs
    - rds

Error

Feb 19 13:02:40 elkserver metricbeat: caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers); error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found
Feb 19 13:02:40 elkserver metricbeat: Exiting: 2 errors: error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found
Feb 19 13:02:40 elkserver metricbeat: caused by: RequestError: send request failed
Feb 19 13:02:40 elkserver metricbeat: caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers); error creating aws metricset: failed to retrieve aws credentials, please check AWS credential in config: EC2RoleRequestError: no EC2 instance role found

Thank you in advance.

2

Hi @atharvak,

Umm, this seems to be failing to connect with the metadata endpoint (timeouts to http://169.254.169.254). Is Metricbeat deployed in AWS?

In any case I wonder why it tries to get the security credentials from this endpoint if they are already being provided in the credential file.

Do you see any other error accessing the credentials file?
If you don't have the credentials file in the default path, you may need to set shared_credential_file to the directory containing the credentials file.

3

Did not understand how it happened, but when I gave absolute path for credentials file using shared_credential_file and provided the credential_profile_name, it started working.
Thank you so much!
Following config might help people with same issue.

- module: aws
  period: 1m
  shared_credential_file: /root/.aws/credentials
  credential_profile_name: elastic-profile
  metricsets:
    - elb
    - usage
2 Likes
4

@atharvak good to see this worked for you :slight_smile:

With what user are you running metricbeat? If this is running as root I think that it should be able to find the credentials file in this path without the need of shared_credential_file.

5

Uh, no. My metricbeat is run by root user only. But when I commented out the shared_credentials_file it started showing the same error again. I don't know why...

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.