Input 'aws-s3' failed with: failed to initialize s3 poller: failed to get AWS region for bucket_arn:

Elastic Stack Beats
1

Hello,

I'm having trouble using the S3 module as described here

I'm authenticating with access and secret key and my .yml begins like this:

filebeat.inputs:

- type: aws-s3
  bucket_arn: arn:aws:s3:::guillens
  number_of_workers: 5
  bucket_list_interval: 300s
  access_key_id: 'myAccessKey'
  secret_access_key: 'mySecretKey'
  endpoint: 's3.us-east-1.amazonaws.com'

When I run my filebeat I get the following error:

Input 'aws-s3' failed with: failed to initialize s3 poller: failed to get AWS region for bucket_arn: exceeded maximum number of attempts, 3, request send failed, Get "https://s3.us-east-1.s3-accesspoint.us-east-1.amazonaws.com/guillens?location=": x509: certificate is valid for s3.amazonaws.com, *.s3.amazonaws.com, *.s3.dualstack.us-east-1.amazonaws.com, s3.dualstack.us-east-1.amazonaws.com, *.s3.us-east-1.amazonaws.com, s3.us-east-1.amazonaws.com, *.s3-control.us-east-1.amazonaws.com, s3-control.us-east-1.amazonaws.com, *.s3-control.dualstack.us-east-1.amazonaws.com, s3-control.dualstack.us-east-1.amazonaws.com, *.s3-accesspoint.us-east-1.amazonaws.com, *.s3-accesspoint.dualstack.us-east-1.amazonaws.com, *.s3.us-east-1.vpce.amazonaws.com, not s3.us-east-1.s3-accesspoint.us-east-1.amazonaws.com	{"id": "D16ABDB47E2E81B3"}

It looks like the issue has to do with this x509: certificate is valid for s3.amazonaws.com but I'm not sure how to resolve this? Couldn't find any info online regarding this.

Anyone know where I get this certificate?

2

Remove the endpoint config. You don't need it unless your using a special AWS environment or 3rd party S3 services.

3

when i do that i get the following:

"unknown endpoint, could not resolve endpoint, partition: \"aws\", service: \"s3\", region: \"\", known: [eu-north-1 us-east-2 s3-external-1 sa-east-1 af-south-1 ap-east-1 ap-south-1 eu-central-1 us-west-2 ap-northeast-2 ap-southeast-1 aws-global us-east-1 ca-central-1 eu-south-1 me-south-1 eu-west-1 us-west-1 eu-west-2 eu-west-3 ap-northeast-1 ap-southeast-2]"}
4

What version of Filebeat are u using? And are u trying to use an S3 access point instead of the standard urls?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.