# AWS vpcflow errors - count not find region configuration, context deadline exceeded
**URL:** https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471
**Category:** Beats
**Tags:** filebeat
**Created:** [March 27, 2020, 11:28pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471 "2020-03-27T23:28:56Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@swisscheese](https://discuss.elastic.co/u/swisscheese)
#### Post date: [March 27, 2020, 11:28pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/1 "2020-03-27T23:28:56Z")
I have vpc flow logs going to an S3 bucket and an SQS notification for any object creation event.
I actually much preferred the logstash method of polling the bucket, for several reasons - mainly the ability to re-index from source easily, use regex matching of file patterns, and general simplicity. The realtime notifications don't provide me with much benefit and just add complexity. I hope to see polling as an option on filebeat some day.
Nonetheless, I've set it up, and I am using an instance role on my filebeat node for access to SQS and S3.
Here is my aws.yml:
- module: aws
vpcflow:
enabled: true
var.queue\_url: [https://sqs.us-west-2.amazonaws.com/](https://sqs.us-west-2.amazonaws.com/)/vpcflow
Although quite a bit of data is being read successfully and I can see it in Kibana, I'm getting a lot of errors and warnings. I have no easy way to tell if it's getting everything, but these messages imply that some may be failing and getting dropped. I can't be certain though, because they're very confusing messages.
2020-03-27T01:27:10.019Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:20.019Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:20.019Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:30.020Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:30.020Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:40.020Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:40.020Z ERROR [s3] s3/input.go:204 failed to receive message from SQS: MissingRegion: could not find region configuration
2020-03-27T01:27:40.887Z ERROR [s3] s3/input.go:475 ReadString failed: context deadline exceeded
2020-03-27T01:27:41.531Z ERROR [s3] s3/input.go:475 ReadString failed: context deadline exceeded
2020-03-27T01:27:41.531Z ERROR [s3] s3/input.go:386 createEventsFromS3Info failed for AWSLogs//vpcflowlogs/us-west-2/2020/03/26/\_vpcflowlogs\_us-west-2\_fl-.log.gz: ReadString failed: conte
xt deadline exceeded
2020-03-27T01:27:41.571Z WARN [s3] s3/input.go:277 Processing message failed, updating visibility timeout
2020-03-27T01:27:41.636Z INFO [s3] s3/input.go:282 Message visibility timeout updated to 300
2020-03-27T01:27:41.794Z WARN [s3] s3/input.go:277 Processing message failed, updating visibility timeout
2020-03-27T01:27:41.799Z INFO [s3] s3/input.go:282 Message visibility timeout updated to 300
2020-03-27T01:27:41.993Z WARN [s3] s3/input.go:277 Processing message failed, updating visibility timeout
2020-03-27T01:27:41.997Z INFO [s3] s3/input.go:282 Message visibility timeout updated to 300
Any ideas?
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [March 30, 2020, 1:54pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/2 "2020-03-30T13:54:42Z")
Hi @swisscheese, thank you for letting us know about your preference on polling method! I will create a github issue to track this.
For the error message you see in the log, I believe it's caused by other filesets that are enabled(by default) in aws module. We do have a github issue to fix this: [https://github.com/elastic/beats/issues/17256](https://github.com/elastic/beats/issues/17256)
If you change `aws.yml` to the config below(assume you are running 7.6 version), you should see a better/cleaner log.
```auto
- module: aws
cloudtrail:
enabled: false
elb:
enabled: false
s3access:
enabled: false
vpcflow:
enabled: true
var.queue_url: https://sqs.us-west-2.amazonaws.com//vpcflow
```
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [April 2, 2020, 11:18pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/4 "2020-04-02T23:18:06Z")
Ahh sorry I missed this error message. Question: seems like this error showed up twice for the same message `AWSLogs/699536110035/vpcflowlogs/us-west-2/2020/03/29/_vpcflowlogs_us-west-2_fl-.log.gz` in 1 second, which is not cool... Did you by any chance changed the `visibility_timeout` param?
```auto
2020-03-30T18:30:44.613Z ERROR [s3] s3/input.go:386 createEventsFromS3Info failed for AWSLogs/699536110035/vpcflowlogs/us-west-2/2020/03/29/_vpcflowlogs_us-west-2_fl-.log.gz: ReadString failed: context deadline exceeded
2020-03-30T18:30:45.207Z ERROR [s3] s3/input.go:386 createEventsFromS3Info failed for AWSLogs/699536110035/vpcflowlogs/us-west-2/2020/03/29/_vpcflowlogs_us-west-2_fl-.log.gz: ReadString failed: context deadline exceeded
```
Do you know if this log file eventually get read by Filebeat? Or it always show in the error message repeatedly?
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [April 2, 2020, 11:19pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/5 "2020-04-02T23:19:07Z")
If you could attach the actual file here, that would be great 🙂
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [April 2, 2020, 11:34pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/6 "2020-04-02T23:34:58Z")
Since it's a context deadline exceeded error, could you try increase setting `var.api_timeout` in `aws.yml` please?
---
### Author:  [@tbuatois](https://discuss.elastic.co/u/tbuatois)
#### Post date: [April 9, 2020, 5:48pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/7 "2020-04-09T17:48:40Z")
Hi,
I am facing exactly the same problem as above.
My configuration :
```auto
- module: aws
cloudtrail:
enabled: false
elb:
enabled: false
s3access:
enabled: false
vpcflow:
enabled: true
# AWS SQS queue url
var.queue_url: https://sqs.eu-west-3.amazonaws.com/$AWSACCOUNID/$QUEUEName
```
I did not change any other values.
My error log :
```auto
Apr 09 19:35:21 ip-10-133-50-250. ******.fr filebeat[30039]: 2020-04-09T19:35:21.769+0200 ERROR [s3] s3/input.go:475 ReadString failed: context deadline exceeded
Apr 09 19:35:21 ip-10-133-50-250.******.fr filebeat[30039]: 2020-04-09T19:35:21.769+0200 ERROR [s3] s3/input.go:386 createEventsFromS3Info failed for AWSLogs/******/vpcflowlogs/eu-west-3/2020/04/09 ******* _vpcflowlogs_eu-west-3_fl-0629b5eccc8f3d0ad_20200409T0120Z_9b923cb5.log.gz: ReadString failed: context deadline exceeded
Apr 09 19:35:43 ip-10-133-50-250. ******.fr filebeat[30039]: 2020-04-09T19:35:43.317+0200 WARN [s3] s3/input.go:277 Processing message failed, updating visibility timeout
Apr 09 19:35:43 ip-10-133-50-250. ******.fr filebeat[30039]: 2020-04-09T19:35:43.357+0200 INFO [s3] s3/input.go:282 Message visibility timeout updated to 300
```
My SQS queue settings :
```auto
VPCFlowLogsQueue:
Type: AWS::SQS::Queue
Properties:
DelaySeconds: 0
MaximumMessageSize: 262144
MessageRetentionPeriod: 345600
VisibilityTimeout: 30
SQS queue policy sqs:* for logstash instance role.
```
Like @swisscheese I have a vpc flow logs data showing up but I cannot be certain that is all of it.
Elasticsearch Version : 7.5.1
Filebeat Version : 7.6.2 (Use default template/dashboard and index)
S3 and SQS Access trough instance profile role on AWS EC2. (s3:\* and sqs:\* on specified resources).
Any thoughts ?
Regards,
Thomas.
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [April 13, 2020, 9:18pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/8 "2020-04-13T21:18:54Z")
Could you give this config a try with increasing `var.api_timeout` please?
```auto
- module: aws
cloudtrail:
enabled: false
elb:
enabled: false
s3access:
enabled: false
vpcflow:
enabled: true
var.queue_url: https://sqs.eu-west-3.amazonaws.com/$AWSACCOUNID/$QUEUEName
var.api_timeout: 720s
var.visibility_timeout: 300s
```
Also since the error message is complaining about file `AWSLogs/ ******/vpcflowlogs/eu-west-3/2020/04/09******* _vpcflowlogs_eu-west-3_fl-0629b5eccc8f3d0ad_20200409T0120Z_9b923cb5.log.gz`, if you can check in Kibana discover (maybe with a filter) to see if there are events from this file please?
Thanks,
Kaiyan
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 11, 2020, 9:18pm UTC](https://discuss.elastic.co/t/aws-vpcflow-errors-count-not-find-region-configuration-context-deadline-exceeded/225471/9 "2020-05-11T21:18:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.