# \[Filebeat\] AWS module & ELB fileset, not parsing message
**URL:** https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690
**Category:** Beats
**Tags:** beats-module, filebeat
**Created:** [October 1, 2020, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690 "2020-10-01T14:00:01Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 1, 2020, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/1 "2020-10-01T14:00:01Z")
Hi there !
I've been setting the `aws` module and the ELB fileset with it, and got it mostly working.
It reads the SQS Queue, then get the matching log on S3 side, but it doesn't parse the Access Log message at all.
Im running Filebeat in K8s, using official image `docker.elastic.co/beats/filebeat:7.9.2`.
Here is my config :
```auto
filebeat.modules:
- module: aws
elb:
enabled: true
var.queue_url: https://sqs.{{ aws_region }}.amazonaws.com/{{ aws_account_id }}/{{ sqs_queue_name }}
var.shared_credential_file: /etc/filebeat/aws_credentials
credential_profile_name: default
var.visibility_timeout: 300s
var.api_timeout: 120s
cloudtrail.enabled: false
cloudwatch.enabled: false
ec2.enabled: false
s3access.enabled: false
vpcflow.enabled: false
output.console:
enabled: true
```
Here is a sample output :
```auto
{
"@timestamp":"2020-10-01T13:38:07.424Z",
"@metadata":{
"beat":"filebeat",
"type":"_doc",
"version":"7.9.2",
"_id":"fa53066972-000002163821",
"pipeline":"filebeat-7.9.2-aws-elb-pipeline"
},
"aws":{
"s3":{
"bucket":{
"name":"???????????",
"arn":"arn:aws:s3:::??????????????"
},
"object.key":"??????/AWSLogs/???/elasticloadbalancing/us-east-1/2020/09/30/???????.log"
}
},
"tags":[
"forwarded"
],
"agent":{
"name":"ip-??-??-??-??.ec2.internal",
"type":"filebeat",
"version":"7.9.2",
"hostname":"ip-??-??-??-??.ec2.internal",
"ephemeral_id":"b132b4bc-5c12-47bc-9495-1e5b00747d80",
"id":"00f8a6fa-f9b4-40bf-b8fa-5cbd42e54a4c"
},
"ecs":{
"version":"1.5.0"
},
"message":"2020-09-30T22:49:17.793407Z aa4ac2f5ea65111eaa5a30e9585490a4 ??.??.??.??:46312 ??.??.??.??:31184 0.00003 0.014323 0.000057 200 200 0 356 \"GET http://????.lan:80/???????/?list-type=2&prefix=repositories%2Fnone%2Fdataset%2Ffull_content&fetch-owner=false HTTP/1.1\" \"aws-sdk-java/1.11.415 Linux/4.14.186-146.268.amzn2.x86_64 OpenJDK_64-Bit_Server_VM/11.0.6+10-LTS java/11.0.6\" - -",
"log":{
"offset":2163821,
"file.path":"https://?????.s3-us-east-1.amazonaws.com/???????/AWSLogs/?????/elasticloadbalancing/us-east-1/2020/09/30/?????.log"
},
"cloud":{
"provider":"aws",
"region":"us-east-1"
},
"event":{
"module":"aws",
"dataset":"aws.elb"
},
"fileset":{
"name":"elb"
},
"service":{
"type":"aws"
},
"input":{
"type":"s3"
}
}
```
As you can see, there are no `aws.elb` fields created, even if `dataset: "aws.elb"` and `"pipeline":"filebeat-7.9.2-aws-elb-pipeline"` are correct.
---
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 1, 2020, 2:08pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/2 "2020-10-01T14:08:47Z")
@Kaiyan_Sheng would you have 5min to look at it ?
I've seen you already answered to similar problems in here 🙂
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [October 1, 2020, 2:45pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/3 "2020-10-01T14:45:56Z")
@Salim_B Hello! Very interesting case! What kind of load balancer is this? Classic or Application load balancer or Network load balancer? We have different grok patterns to parse different kinds of elb logs so probably we are hitting something we don't support or a bug in the pattern 🙂
---
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 1, 2020, 5:11pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/4 "2020-10-01T17:11:34Z")
@Kaiyan_Sheng thx for that quick answer 🙂
The logs are from `classic` ELB. And the one in the exemple is a `TCP` one.
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [October 1, 2020, 7:30pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/5 "2020-10-01T19:30:51Z")
Thank you! I will try to reproduce this problem locally with the sample log:
```auto
2020-09-30T22:49:17.793407Z aa4ac2f5ea65111eaa5a30e9585490a4 ??.??.??.??:46312 ??.??.??.??:31184 0.00003 0.014323 0.000057 200 200 0 356 \"GET http://????.lan:80/???????/?list-type=2&prefix=repositories%2Fnone%2Fdataset%2Ffull_content&fetch-owner=false HTTP/1.1\" \"aws-sdk-java/1.11.415 Linux/4.14.186-146.268.amzn2.x86_64 OpenJDK_64-Bit_Server_VM/11.0.6+10-LTS java/11.0.6\" - -
```
---
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 1, 2020, 8:44pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/6 "2020-10-01T20:44:04Z")
Alright, many thanks !
If needed I could provide in private some real lines without replacing our IPs and FQDNs by `?` 😃
---
### Author:  [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [October 5, 2020, 1:11pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/7 "2020-10-05T13:11:08Z")
Hi @Salim_B I tried to reproduce it locally and seems like there is some incompatibility between your sample log and the grok pattern we have for ELB. Do you mind creating a github issue for this please? I will add it to our roadmap 🙂 Thank you!!!
---
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 5, 2020, 2:03pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/8 "2020-10-05T14:03:44Z")
Done here : [https://github.com/elastic/beats/issues/21522](https://github.com/elastic/beats/issues/21522) 😉
---
### Author:  [@Salim\_B](https://discuss.elastic.co/u/Salim_B)
#### Post date: [October 5, 2020, 2:05pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/9 "2020-10-05T14:05:34Z")
@Kaiyan_Sheng any chance to see a fix appear in a 7.9.3 release ? 👼
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 2, 2020, 4:05pm UTC](https://discuss.elastic.co/t/filebeat-aws-module-elb-fileset-not-parsing-message/250690/10 "2020-11-02T16:05:44Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.