# How to parse AWS Cloudfront logs
**URL:** https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690
**Category:** Beats
**Tags:** docker, filebeat
**Created:** [January 6, 2022, 8:28pm UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690 "2022-01-06T20:28:29Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@leo-baltus](https://discuss.elastic.co/u/leo-baltus)
#### Post date: [January 6, 2022, 8:28pm UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/1 "2022-01-06T20:28:29Z")
Tring to get filebeat to parse AWS Cloudfront logs.
SQS seem to work. but once in ES I see:
```auto
Provided Grok expressions do not match field value: [2022-01-06\t20:06:28\tFRA56-C2\t1729\t3.125.241.170\tGET\tdtc81dn1qkg0w.cloudfront.net\t/.well-known/...
```
filebeat.yml:
```auto
cloud.id: "${ELASTICSEARCH_FILEBEAT_CLOUD_ID}"
cloud.auth: "${ELASTICSEARCH_FILEBEAT_CLOUD_AUTH}"
filebeat.modules:
- module: aws
s3access:
enabled: true
var.queue_url: "${SQS_QUEUE}"
var.access_key_id: "${AWS_ACCESS_KEY_ID}"
var.secret_access_key: "${AWS_SECRET_ACCESS_KEY}"
```
Tried `elb` and `aws-s3` to no avail.
I am running filebeat in k8s dockerimage: `docker.elastic.co/beats/filebeat:7.16.2`
Cloudfront logs are documented
[here](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html)
Sample log:
```auto
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version fle-status fle-encrypted-fields c-port time-to-first-byte x-edge-detailed-result-type sc-content-type sc-content-len sc-range-start sc-range-end
2022-01-06 14:37:55 MUC50-P1 405 3.125.241.170 GET dtc819989700w.cloudfront.net /versioninfo/version 200 - curl/7.68.0 - - Miss aekvP2AgpYb1uYYV1dd8lCjcUwhEJGQucpuvgzafdJ1XXscToIlnPg== xxxxxxxxxx.yyyyyyyyy.nl https 61 0.051 - TLSv1.3 TLS_AES_128_GCM_SHA256 Miss HTTP/2.0 - - 49716 0.051 Miss text/plain;%20charset=utf-8 - - -
```
Any help or pointers are welcome.
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [January 7, 2022, 3:11am UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/2 "2022-01-07T03:11:41Z")
CloudFront logs and S3 access logs are not the same. You'll have to use a manual`aws-s3` input and define a custom ingest pipeline to parse it.
---
### Author:  [@leo-baltus](https://discuss.elastic.co/u/leo-baltus)
#### Post date: [January 7, 2022, 10:05am UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/3 "2022-01-07T10:05:15Z")
Thank you. Any pointers on examples or docs? there's not a great deal of information on this it seems.
---
### Author:  [@legoguy1000](https://discuss.elastic.co/u/legoguy1000)
#### Post date: [January 7, 2022, 3:05pm UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/4 "2022-01-07T15:05:28Z")
[AWS S3 input | Filebeat Reference [7.16] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-aws-s3.html) for how to setup the input. I built this pipeline last night, [[AWS] Add CloudFront logs datastream by legoguy1000 · Pull Request #2476 · elastic/integrations · GitHub](https://github.com/elastic/integrations/pull/2476/files#diff-11a70ab03f957c703207b4a6088c9e46da385f0ce3f32c1a0887f638210fe1f4). You'll just have to point filebeat to this pipeline in ES either at the input or the out.
---
### Author:  [@leo-baltus](https://discuss.elastic.co/u/leo-baltus)
#### Post date: [January 8, 2022, 9:49am UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/5 "2022-01-08T09:49:04Z")
I'm very impressed! Hope this PR gets accepted soon.
Thank you.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 5, 2022, 11:49am UTC](https://discuss.elastic.co/t/how-to-parse-aws-cloudfront-logs/293690/6 "2022-02-05T11:49:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.