# Filebeat AWS Module Errors and Missing Logs
**URL:** https://discuss.elastic.co/t/filebeat-aws-module-errors-and-missing-logs/244503
**Category:** Beats
**Tags:** beats-module, filebeat
**Created:** [August 11, 2020, 7:36am UTC](https://discuss.elastic.co/t/filebeat-aws-module-errors-and-missing-logs/244503 "2020-08-11T07:36:17Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@johnnywalker](https://discuss.elastic.co/u/johnnywalker)
#### Post date: [August 11, 2020, 7:36am UTC](https://discuss.elastic.co/t/filebeat-aws-module-errors-and-missing-logs/244503/1 "2020-08-11T07:36:17Z")
Prior to our 7.8 migration, we used our own cloudtrail worker(golang + logstash) but wanted to adopt the aws filebeat module for ECS conversion.
Upon doing so I notice a significant drop in log volume(500GB -\> 30GB), missing actions(ConsoleLogin), additional ECS fields(such as crowdstrike.event.StartTimestamp), and a significant amount of MissingRegion errors. Am I missing anything? Is there any additional setup requirements? Is the ingest pipeline not working? We are using ECK and just a standard filebeat deployment, not using the Beats resource kind, in kubernetes.
Any help would be greatly appreciated!
Numerous:
`s3/input.go:206 SQS ReceiveMessageRequest failed: MissingRegion: could not find region configuration`
Without S3 Input(filebeat.inputs):
`s3/input.go:538 decode json failed for 'AWSLogs/000000000000/CloudTrail-Digest/us-east-1/2020/07/22/000000000000_CloudTrail-Digest_us-east-1_redacted-logging_us-east-1_20200722T030000Z.json.gz' from S3 bucket 'redacted-logging', skipping this file: json: cannot unmarshal string into Go value of type []interface {}`
filebeat.yml:
```auto
setup.template.enabled: true
setup.ilm.enabled: false
setup.template.overwrite: true
setup.template.name: "cloudtrail"
setup.template.pattern: "cloudtrail-*"
setup.template.settings:
index.number_of_shards: 3
index.number_of_replicas: 1
filebeat.inputs:
- type: s3
queue_url: "https://sqs.us-east-1.amazonaws.com/000000000000/logging-notifications"
expand_event_list_from_field: Records
filebeat.modules:
- module: aws
cloudtrail:
enabled: true
var.queue_url: https://sqs.us-east-1.amazonaws.com/000000000000/logging-notifications
var.visibility_timeout: 300s
elb.enabled: false
s3access.enabled: false
vpcflow.enabled: false
output.elasticsearch:
hosts: ['https://elasticsearch.elastic-system.svc.cluster.local:9200']
ssl.certificate_authorities: ["/usr/share/filebeat/ca.crt"]
ssl.verification_mode: none
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
index: "cloudtrail-%{+yyyy.MM.dd}"
pipeline: filebeat-7.8.1-aws-cloudtrail-pipeline
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 8, 2020, 9:36am UTC](https://discuss.elastic.co/t/filebeat-aws-module-errors-and-missing-logs/244503/2 "2020-09-08T09:36:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.