# Using Filebeat to fetch CrowdStrike Falcon Data Replicator (FDR) logs with S3 SQS
**URL:** https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395
**Category:** Beats
**Tags:** filebeat
**Created:** [June 18, 2021, 2:03pm UTC](https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395 "2021-06-18T14:03:41Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@meatwad](https://discuss.elastic.co/u/meatwad)
#### Post date: [June 18, 2021, 2:03pm UTC](https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395/1 "2021-06-18T14:03:41Z")
I'm interested in using Filebeat to fetch CrowdStrike Falcon Data Replicator (FDR) logs with the aws-s3 plugin, and use its parallel processing functionality due to the sheer volume of data FDR produces. But I'm running into issues just getting one filebeat instance to successfully fetch the data. Here's my config:
```auto
filebeat.inputs:
- type: aws-s3
enabled: true
queue_url: https://sqs.us-west-1.amazonaws.com/$CUSTOMER_ID/$QUEUE
shared_credential_file: /etc/filebeat/fdr_credentials
max_number_of_messages: 10
#============================= Filebeat modules ===============================
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 1
#================================ General =====================================
tags: ["FDR"]
#================================ Outputs =====================================
output.logstash:
hosts: ["localhost:5044"]
ssl.certificate_authorities: ["/etc/filebeat/ssl/ca.crt.pem"]
ssl.certificate: "/etc/filebeat/ssl/beats.crt.pem"
ssl.key: "/etc/filebeat/ssl/beats.key.pem"
#================================ Logging =====================================
logging.level: info
```
But when I run it, I get the following:
```auto
404 - Not Found
404 - Not Found
{"queue_url": "https://sqs.us-west-1.amazonaws.com/$CUSTOMER_ID/$QUEUE", "region": "us-west-1"}
2021-06-18T13:56:11.834Z INFO beater/filebeat.go:515 Stopping filebeat
2021-06-18T13:56:11.834Z INFO beater/crawler.go:148 Stopping Crawler
2021-06-18T13:56:11.834Z INFO beater/crawler.go:158 Stopping 1 inputs
2021-06-18T13:56:11.834Z INFO cfgfile/reload.go:227 Dynamic config reloader stopped
2021-06-18T13:56:11.834Z INFO [crawler] beater/crawler.go:163 Stopping input: 12213181681190882779
2021-06-18T13:56:11.834Z INFO [input.aws-s3] compat/compat.go:132 Input 'aws-s3' stopped
2021-06-18T13:56:11.834Z INFO beater/crawler.go:178 Crawler stopped
2021-06-18T13:56:11.834Z INFO [registrar] registrar/registrar.go:132 Stopping Registrar
2021-06-18T13:56:11.834Z INFO [registrar] registrar/registrar.go:166 Ending Registrar
2021-06-18T13:56:11.835Z INFO [registrar] registrar/registrar.go:137 Registrar stopped
2021-06-18T13:56:11.836Z ERROR [input.aws-s3] awss3/collector.go:99 SQS ReceiveMessageRequest failed: EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make Client request
caused by:
```
I know the creds and the SQS queue are right because if I use 'aws' on the command line, I can see all of the dirs and gzipped files in the bucket. What am I doing wrong?
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [June 20, 2021, 1:46pm UTC](https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395/2 "2021-06-20T13:46:14Z")
> [@meatwad](#):
>
> `ERROR [input.aws-s3] awss3/collector.go:99 SQS ReceiveMessageRequest failed: EC2RoleRequestError: no EC2 instance role found`
What credentials are you using? You haven't defined anything in the input config so it looks like it's using the default which u don't have set up.
---
### Author:  [@meatwad](https://discuss.elastic.co/u/meatwad)
#### Post date: [June 21, 2021, 8:37pm UTC](https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395/3 "2021-06-21T20:37:31Z")
> [@meatwad](#):
>
> `shared_credential_file: /etc/filebeat/fdr_credentials`
Thanks for your reply. I'm using the creds that are defined in "shared\_credential\_file: /etc/filebeat/fdr\_credentials".
I think my issue is that I'm just looking for content in the gzipped files that are being written to the S3 bucket, not cloudtrail, cloudwatch, ec2, elb, s3access, or vpcflow logs.
Thanks again
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 19, 2021, 10:38pm UTC](https://discuss.elastic.co/t/using-filebeat-to-fetch-crowdstrike-falcon-data-replicator-fdr-logs-with-s3-sqs/276395/4 "2021-07-19T22:38:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.