# Filebeat AWS S3 Module not working **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 30, 2021, 11:33am UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781 "2021-03-30T11:33:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![bapa](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@bapa](https://discuss.elastic.co/u/bapa)\ **Post date:** [March 30, 2021, 11:33am UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/1 "2021-03-30T11:33:36Z")
Hi Team, We are facing the filebeat aws s3 module issue in version 7.10 ,7.10.1, 7.11.0, 7.11.1, 7.11.2, 7.12.0 We have deployed the filebeat in k8s environment. And the ELK with ECK operator. While we tried to use the filebeat aws s3 module to get the data SQS queue we are getting the below error. ERROR: 2021-03-30T10:35:42.401Z ERROR [input.s3] s3/collector.go:107 SQS ReceiveMessageRequest failed: InvalidClientTokenId: The security token included in the request is invalid status code: 403, request id: XXXXXX {"queue\_url": "XXXXXXX", "region": "XXXXX"} Note: We are not using the aws credentials we are using the IAM role to access the SQS and S3. As we have verified we are all clear in IAM permission and we dont find any issue with IAM here. Kindly provide the guides to overcome this issue ASAP ---
**Author:** ![bapa](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@bapa](https://discuss.elastic.co/u/bapa)\ **Post date:** [March 30, 2021, 12:39pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/2 "2021-03-30T12:39:04Z")
@Kaiyan_Sheng ---
**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\ **Post date:** [March 30, 2021, 4:17pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/3 "2021-03-30T16:17:55Z")
Thanks for creating the issue here! Using `role_arn` still requires access key id and secret access key to be under the shared credential file `[default]` section. Here is the link for how to create the shared credentials file: [Create a shared credentials file - Amazon Simple Email Service](https://docs.aws.amazon.com/ses/latest/DeveloperGuide/create-shared-credentials-file.html). This file by default is `~/.aws/credentials` for Linux and macOS. Do you have this file with the correct credentials under `[default]` section? ---
**Author:** ![bapa](https://avatars.discourse-cdn.com/v4/letter/b/3ab097/32.png) [@bapa](https://discuss.elastic.co/u/bapa)\ **Post date:** [March 30, 2021, 4:34pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/4 "2021-03-30T16:34:32Z")
Hy kaiyan, Earlier we dont have above setup which u have mentioned but worked. We used only role\_arn ---
**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\ **Post date:** [April 5, 2021, 3:19pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/5 "2021-04-05T15:19:38Z")
Hey @bapa I'm also looking at adding support for `role_arn` with given `aws_access_key_id` and `aws_secret_access_key` (without the shared credential file). So only `role_arn` worked before but not working now? ---
**Author:** ![nugroho-expereo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nugroho-expereo/32/78333_2.png) [@nugroho-expereo](https://discuss.elastic.co/u/nugroho-expereo)\ **Post date:** [April 5, 2021, 8:03pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/6 "2021-04-05T20:03:31Z")
Hi @Kaiyan_Sheng I also experienced similar issue with the latest Filebeat v7.12. It seems there was recent change that breaks IAM role usage attached to EC2 ( [Fleet AWS Cloudtrail integration stops working after upgrading elastic agent to 7.12.0](https://discuss.elastic.co/t/fleet-aws-cloudtrail-integration-stops-working-after-upgrading-elastic-agent-to-7-12-0/269120)). What you are suggesting (specify access key or credentials file) is valid for beats deployment outside AWS environment but that is against best practices for deployment within AWS (EC2 or ECS task). See [Configuring the AWS SDK for Go - AWS SDK for Go](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) AWS suggests to configure IAM permission in the following order of preference: 1. Use IAM Task Role for ECS task 2. Use IAM role attached to EC2 (technically it is a role associated to IAM instance profile that is attached to EC2) 3. Use credentials file 4. Use environment variables With option 2 above it is really convenient and more secure because there's no access key to manage or to rotate, only need to attach a role to an instance. AWS CLI and SDK supports all those options. For Filebeat configuration I think Elastic should implement in such a way if none are configured (all blank) then it will fallback to attached Task Role or IAM instance role or introducing a boolean configuration for attached IAM Role. ---
**Author:** ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)\ **Post date:** [April 6, 2021, 3:42pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/7 "2021-04-06T15:42:30Z")
Thanks @nugroho-expereo for your suggestion!! We did add the `var.role_arn` config option in the [aws module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-aws.html#_module_configuration) but right now it only works when credentials are set in the shared credential file under the default profile. Will investigate more here! Thank you again!! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 4, 2021, 5:43pm UTC](https://discuss.elastic.co/t/filebeat-aws-s3-module-not-working/268781/8 "2021-05-04T17:43:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.