# 7.9.2 FileBeat CloudTrail AWS Module Changes Break Log Collection

**URL:** https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661
**Category:** Beats
**Tags:** filebeat
**Created:** [October 10, 2020, 11:32pm UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661 "2020-10-10T23:32:46Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![djcullen](https://avatars.discourse-cdn.com/v4/letter/d/bcef8e/32.png) [@djcullen](https://discuss.elastic.co/u/djcullen)
#### Post date: [October 10, 2020, 11:32pm UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661/1 "2020-10-10T23:32:46Z")

</div>

It appears that the changes made to the FileBeat AWS module in 7.9.2 (specifically [21086](https://github.com/elastic/beats/pull/21086)) break the module for anyone whose CloudTrail S3 object key does not conform to the following regex:  
`^AWSLogs/\d+/CloudTrail/`

My organization's CloudTrail logs follow the naming convention below which doesn't conform to the baked-in regex above and results in FileBeat simply deleting all messages in my SQS queue when using the AWS module:  
mgmt/AWSLogs/8\*\*\*\*\*\*\*\*\*\*/CloudTrail/

I can disable the AWS module and declare the following regex using file\_selectors in my filebeat.yml, but I have not been able to do so successfully within aws.yml (I've tried creating a var.file\_selectors parameter, but it is just ignored):  
`mgmt\/AWSLogs\/\d+\/CloudTrail\/.*`

I only want to collect the mgmt logs and not the data or digest logs, and this regex works for that purpose. I'd prefer to use the AWS module. Is there something I'm missing that will allow me to change the default regex used in the AWS module?

---

<div class="post-metadata">

### Author: ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)
#### Post date: [October 12, 2020, 8:44am UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661/2 "2020-10-12T08:44:35Z")

</div>

I'm afraid you won't be able to fix it without using a custom config based on the S3 input (and file selectors).

What you can do is to manually adjust the ingest pipeline in Elasticsearch.

---

<div class="post-metadata">

### Author: ![djcullen](https://avatars.discourse-cdn.com/v4/letter/d/bcef8e/32.png) [@djcullen](https://discuss.elastic.co/u/djcullen)
#### Post date: [October 12, 2020, 6:34pm UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661/3 "2020-10-12T18:34:26Z")

</div>

Hi Marcin. I was hoping for a more efficient solution to this issue, but I appreciate the response and workaround. I'll keep the AWS module disabled and hope for some flexibility down the road. Thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 9, 2020, 8:35pm UTC](https://discuss.elastic.co/t/7-9-2-filebeat-cloudtrail-aws-module-changes-break-log-collection/251661/4 "2020-11-09T20:35:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
