I'm trying to use aws-cloudwatch input with Filebeat 7.15.2, because I don't want to add every lambda log group separately. I've tried following input config in my filebeat.yml:
filebeat.inputs:
- type: aws-cloudwatch
log_group_name_prefix: /aws/lambda
region_name: eu-west-1
role_arn: <role-arn-here>
However, filebeat only starts an input worker for one log group:
2021-11-15 09:55:54
2021-11-15T07:55:54.919Z INFO [aws-cloudwatch] awscloudwatch/input.go:154 aws-cloudwatch input worker for log group: '/aws/lambda/<lambda-name>' has started
2021-11-15 09:55:54
2021-11-15T07:55:54.919Z INFO [crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 1
2021-11-15 09:55:54
2021-11-15T07:55:54.919Z INFO [crawler] beater/crawler.go:141 Starting input (ID: 18216316265758894950)
2021-11-15 09:55:54
2021-11-15T07:55:54.919Z INFO [aws-cloudwatch] awscloudwatch/input.go:127 Initialized AWS CloudWatch input.
2021-11-15 09:55:54
2021-11-15T07:55:54.918Z WARN [cfgwarn] awscloudwatch/input.go:81 BETA: aws-clouwatch input type is used
I've tried building filebeat locally and added some logging and the prefix itself works i.e. logGroupNames in the input does contain all the log groups in my account, but filebeat only starts worker for one.
According to this Cherry-pick #26187 to 7.x: Add log_group_name_prefix config option for aws-cloudwatch input by kaiyan-sheng · Pull Request #26527 · elastic/beats · GitHub my config should work, is this a bug or is there something wrong with the filebeat.yml?