"filter_pattern" parsing AWS ALB-logs | "Invalid subscription filter pattern" in CF

Hi,

Running a Lambda created by AWS to fetch ALB-access logs from S3 and import them to CloudWatch;

However when viewing the log-group there are only the fields "Timestamp" and "Message" where "Message" actually is all the different fields I can query through Athena;
https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html
(see "Fields" in the link above for the different fields)

However I found the actual regex that Athena uses to parse the "Message"-field here:
https://docs.aws.amazon.com/athena/latest/ug/application-load-balancer-logs.html

When I tried it on the raw "Message"-field it was able to group that info into different parts and I then thought of adding it to Functionbeat in the config;

filter_pattern: '([^ ]) ([^ ]) ([^ ]) ([^ ]):([0-9]) ([^ ]):- ([-.0-9]) ([-.0-9]) ([-.0-9]) (|[-0-9]) (-|[-0-9]) ([-0-9]) ([-0-9]) "([^ ]) ([^ ]) (- |[^ ])" "([^"])" ([A-Z0-9-]+) ([A-Za-z0-9.-]) ([^ ]) "([^"])" "([^"])" "([^"])" ([-.0-9]) ([^ ]) "([^"])" "([^"])"($| "[^ ]")(.)'

I've escaped the regex by putting it inbetween single quoutes but I this error in CF when trying to update my function with this;

Invalid subscription filter pattern (Service: AWSLogs; Status Code: 400; Error Code: InvalidParameterException; Request ID: xxxxxxxxxxxxx)

The following resource(s) failed to update: [myfunction].

Error occurred while GetObject. S3 Error Code: NoSuchKey. S3 Error Message: The specified key does not exist. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: xxxxxxxxxxxxx)

The following resource(s) failed to update: [all_resources].

Grateful for any help in what I am doing wrong :slight_smile:

Best Regards - TheSwede86

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.