Filebeat AWS Module S3 input error queueURL is not in format

Trying to use filebeat to monitor my AWS resources in ISO environment. It looks like the AWS standard endpoint is hard coded.

my filebeat.yml

- type: s3
  visibility_timeout: 300s


ERROR	[s3]	s3/input.go:93 Input 'S3' failed with: getRegionFromQueueURL failed: queueURL is not in format: https://sqs.{REGION_ENDPOINT}{ACCOUNT_NUMBER}/{QUEUE_NAME} 

I am also behind a proxy and dont see the traffic hitting the proxy server, not sure if this is related.

Anything on this???


I think you are right. For some reason the Module expects to find the URL in this format. beats/collector.go at 616266fa286f0ac4408348fcd4da7da2358182b1 · elastic/beats · GitHub

I'm not sure what would be different with your case or if it's just that the URL format is different. Most probably we just need to adjust the code so as to cover cases like yours.

@Kaiyan_Sheng do you think this would be an enhancement request?

Yeah it looks like it is hardcoded to only work if the endpoint is in AWS Commercial with as the endpoint, it looks for that in the code. we would need it to be a variable so instead of looking for say
It should be something like
Allowing you to set the endpoint to whatever AWS ENV you are in so in our case
This is probably also hurting the people in Gov cloud and anything else that does not point to