@Ayush_Mathur
AWS_DEFAULT_REGION
env variable is not read by Filebeat.
Currently where is no possibility to set an AWS region for Filebeat S3 input.
This is taken care by setting it initially to us-east-1
in the S3 client used by Filebeat, that does then a first request to get the bucket region and then override the region of the S3 client accordingly.
This first request is indeed the one that fails for you.
Please try running the following aws cli command:
aws s3api get-bucket-location --debug --bucket es-backup-xxxxx --endpoint-url https://s3.us-east-1.amazonaws.com
this is what Filebeat does. Provide the IAM user or role that Filebeat uses.