# Filebeat Parsing issue with module aws
**URL:** https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374
**Category:** Beats
**Tags:** beats-module, filebeat
**Created:** [April 27, 2021, 12:44pm UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374 "2021-04-27T12:44:04Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@AJ18](https://discuss.elastic.co/u/AJ18)
#### Post date: [April 27, 2021, 12:44pm UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/1 "2021-04-27T12:44:04Z")
Hi,
I had set up filebeat agent in my AWS account to ship logs from an S3 bucket. The s3 bucket contains Cloudtrail logs and VPC Flow Logs and has Access logging enabled.
The file 'filebeat/modules.d/aws.yml' was modified to enable the filesets Cloudtrail, Cloudwatch, VPCFlowlogs and s3access and add the corresponding SQS queue ARN along with the required credentials.
The data is flowing in and is being successfully received from Filebeat AWS module in Elasticsearch. However, the data is incorrectly mapped in the logs.
I can see vpc flow logs associated with Cloudwatch dataset and Cloudtrail datasets, similarly, Cloudtrail logs are being tagged as s3access fileset (Please refer to the attached image).
Please help understand how the data is selected by Filebeat filesets and how the mapping can be corrected.
---
### Author:  [@Marius\_Iversen](https://discuss.elastic.co/u/Marius_Iversen)
#### Post date: [April 27, 2021, 9:33pm UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/2 "2021-04-27T21:33:11Z")
Have you tried the AWS specific dashboards under the Dashboards tab on the left? That might show you something more specifically. All filters you want can be applied both on your screen and in each dashboard 🙂
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [April 28, 2021, 12:55am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/3 "2021-04-28T00:55:00Z")
It looks like u may have a mismatch of the filesets and the data. Are the correct queues set to the correct filesets?
---
### Author:  [@AJ18](https://discuss.elastic.co/u/AJ18)
#### Post date: [April 28, 2021, 3:04am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/4 "2021-04-28T03:04:59Z")
Yes Marius. I have explored the Dashboards but didn't find anything significant that could help. The issue is with the incorrect mapping between data and filesets.
Thank you!
---
### Author:  [@AJ18](https://discuss.elastic.co/u/AJ18)
#### Post date: [April 28, 2021, 3:10am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/5 "2021-04-28T03:10:06Z")
Hi Alex
Thanks for your response.
All the logs are stored in one centralised S3 bucket and have the same queue set to receive event notifications on new objects ingestion in the bucket.
Does filebeat requires separate buckets and queues for data from different services?
Because my use case requires me to have all the logs in a single bucket.
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [April 28, 2021, 6:34pm UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/6 "2021-04-28T18:34:00Z")
Because they are separate filesets each fileset is expecting only to receive its specific logs so i suspect you need different buckets/queues. Are you seeing the same log/event show up multiple times as part of the different filesets?
---
### Author:  [@AJ18](https://discuss.elastic.co/u/AJ18)
#### Post date: [April 29, 2021, 5:45am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/7 "2021-04-29T05:45:10Z")
No Alex, the logs are not repeated as part of different filesets.
Do you think I can find any documentation on this for more clarity on the requirement of separate buckets and queues?
That could be a problem, given my use case
Really appreciate your help on this. Thanks!
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [April 29, 2021, 9:43am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/8 "2021-04-29T09:43:56Z")
I'm not necessarily seeing anything in the docs saying they have to be separate. Are u using the same queue for all the filesets? If so that is probably the issue. Idk if u can create separate queues for a single bucket for different data.
---
### Author:  [@AJ18](https://discuss.elastic.co/u/AJ18)
#### Post date: [April 30, 2021, 11:13am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/9 "2021-04-30T11:13:04Z")
Yes, I was using the same queue for all the data.
I'll try it with different queues as well.
Thanks Alex!
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [May 7, 2021, 10:36am UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/10 "2021-05-07T10:36:29Z")
Did different queues solve this?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 4, 2021, 12:36pm UTC](https://discuss.elastic.co/t/filebeat-parsing-issue-with-module-aws/271374/11 "2021-06-04T12:36:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.