AWS vpcflow errors - count not find region configuration, context deadline exceeded

Elastic Stack Beats
7

Hi,

I am facing exactly the same problem as above.

My configuration :

- module: aws
  cloudtrail:
    enabled: false
  elb:
    enabled: false
  s3access:
    enabled: false
  vpcflow:
    enabled: true
    # AWS SQS queue url
    var.queue_url: https://sqs.eu-west-3.amazonaws.com/$AWSACCOUNID/$QUEUEName

I did not change any other values.

My error log :

Apr 09 19:35:21 ip-10-133-50-250.******.fr filebeat[30039]: 2020-04-09T19:35:21.769+0200        ERROR        [s3]        s3/input.go:475        ReadString failed: context deadline exceeded
Apr 09 19:35:21 ip-10-133-50-250.******.fr filebeat[30039]: 2020-04-09T19:35:21.769+0200        ERROR        [s3]        s3/input.go:386        createEventsFromS3Info failed for AWSLogs/******/vpcflowlogs/eu-west-3/2020/04/09*******_vpcflowlogs_eu-west-3_fl-0629b5eccc8f3d0ad_20200409T0120Z_9b923cb5.log.gz: ReadString failed: context deadline exceeded
Apr 09 19:35:43 ip-10-133-50-250.******.fr filebeat[30039]: 2020-04-09T19:35:43.317+0200        WARN        [s3]        s3/input.go:277        Processing message failed, updating visibility timeout
Apr 09 19:35:43 ip-10-133-50-250.******.fr filebeat[30039]: 2020-04-09T19:35:43.357+0200        INFO        [s3]        s3/input.go:282        Message visibility timeout updated to 300

My SQS queue settings :

VPCFlowLogsQueue:
    Type: AWS::SQS::Queue
    Properties:
      DelaySeconds: 0
      MaximumMessageSize: 262144
      MessageRetentionPeriod: 345600 
      VisibilityTimeout: 30

SQS queue policy sqs:* for logstash instance role.

Like @swisscheese I have a vpc flow logs data showing up but I cannot be certain that is all of it.

Elasticsearch Version : 7.5.1
Filebeat Version : 7.6.2 (Use default template/dashboard and index)

S3 and SQS Access trough instance profile role on AWS EC2. (s3:* and sqs:* on specified resources).

Any thoughts ?

Regards,
Thomas.