Hi @Kaiyan_Sheng I also experienced similar issue with the latest Filebeat v7.12. It seems there was recent change that breaks IAM role usage attached to EC2 ( Fleet AWS Cloudtrail integration stops working after upgrading elastic agent to 7.12.0).
What you are suggesting (specify access key or credentials file) is valid for beats deployment outside AWS environment but that is against best practices for deployment within AWS (EC2 or ECS task).
See Configuring the AWS SDK for Go - AWS SDK for Go
AWS suggests to configure IAM permission in the following order of preference:
- Use IAM Task Role for ECS task
- Use IAM role attached to EC2 (technically it is a role associated to IAM instance profile that is attached to EC2)
- Use credentials file
- Use environment variables
With option 2 above it is really convenient and more secure because there's no access key to manage or to rotate, only need to attach a role to an instance.
AWS CLI and SDK supports all those options.
For Filebeat configuration I think Elastic should implement in such a way if none are configured (all blank) then it will fallback to attached Task Role or IAM instance role or introducing a boolean configuration for attached IAM Role.