# "filter\_pattern" parsing AWS ALB-logs | "Invalid subscription filter pattern" in CF

**URL:** <https://discuss.elastic.co/t/filter-pattern-parsing-aws-alb-logs-invalid-subscription-filter-pattern-in-cf/195513>\
**Category:** Beats\
**Tags:** functionbeat\
**Created:** [August 16, 2019, 2:29pm UTC](https://discuss.elastic.co/t/filter-pattern-parsing-aws-alb-logs-invalid-subscription-filter-pattern-in-cf/195513 "2019-08-16T14:29:14Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheSwede86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theswede86/32/52444_2.png) [@TheSwede86](https://discuss.elastic.co/u/TheSwede86)\
**Post date:** [August 16, 2019, 2:29pm UTC](https://discuss.elastic.co/t/filter-pattern-parsing-aws-alb-logs-invalid-subscription-filter-pattern-in-cf/195513/1 "2019-08-16T14:29:14Z")

</div>

Hi,

Running a Lambda created by AWS to fetch ALB-access logs from S3 and import them to CloudWatch;

> **[GitHub - amazon-archives/cloudwatch-logs-centralize-logs: Sample code - A...](https://github.com/amazon-archives/cloudwatch-logs-centralize-logs)**
>
> Sample code - A Lambda function that helps in centralizing logs from Elastic Load Balancing (ELB) using Amazon S3 bucket triggers. - GitHub - amazon-archives/cloudwatch-logs-centralize-logs: Sample...

However when viewing the log-group there are only the fields "Timestamp" and "Message" where "Message" actually is all the different fields I can query through Athena;

> **[Access logs for your Application Load Balancer - Elastic Load Balancing](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html)**
>
> Learn how to monitor your Application Load Balancer using access logs provided by Elastic Load Balancing.

(see "Fields" in the link above for the different fields)

However I found the actual regex that Athena uses to parse the "Message"-field here:

> **[Querying Application Load Balancer logs - Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/application-load-balancer-logs.html)**
>
> Use Athena to read Application Load Balancer logs.

When I tried it on the raw "Message"-field it was able to group that info into different parts and I then thought of adding it to Functionbeat in the config;

> filter\_pattern: '([^]_) ([^]_) ([^]_) ([^]_):([0-9]_) ([^]_):- ([-.0-9]_) ([-.0-9]_) ([-.0-9]_) (|[-0-9]_) (-|[-0-9]_) ([-0-9]_) ([-0-9]_) "([^]_) ([^]_) (- |[^]_)" "([^"]_)" ([A-Z0-9-]+) ([A-Za-z0-9.-]_) ([^]_) "([^"]_)" "([^"]_)" "([^"]_)" ([-.0-9]_) ([^]_) "([^"]_)" "([^"]_)"($| "[^]_")(._)'

I've escaped the regex by putting it inbetween single quoutes but I this error in CF when trying to update my function with this;

> Invalid subscription filter pattern (Service: AWSLogs; Status Code: 400; Error Code: InvalidParameterException; Request ID: xxxxxxxxxxxxx)
> 
> The following resource(s) failed to update: [myfunction].
> 
> Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: xxxxxxxxxxxxx)
> 
> The following resource(s) failed to update: [all\_resources].

Grateful for any help in what I am doing wrong 🙂

Best Regards - TheSwede86

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 6, 2019, 4:29am UTC](https://discuss.elastic.co/t/filter-pattern-parsing-aws-alb-logs-invalid-subscription-filter-pattern-in-cf/195513/2 "2019-09-06T04:29:19Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.