Filebeat cloudwatch input log_group_name_prefix only starts one input

Elastic Stack Beats
1

I'm trying to use aws-cloudwatch input with Filebeat 7.15.2, because I don't want to add every lambda log group separately. I've tried following input config in my filebeat.yml:


filebeat.inputs:
- type: aws-cloudwatch
  log_group_name_prefix: /aws/lambda
  region_name: eu-west-1
  role_arn: <role-arn-here>

However, filebeat only starts an input worker for one log group:

2021-11-15 09:55:54	
2021-11-15T07:55:54.919Z	INFO	[aws-cloudwatch]	awscloudwatch/input.go:154	aws-cloudwatch input worker for log group: '/aws/lambda/<lambda-name>' has started
2021-11-15 09:55:54	
2021-11-15T07:55:54.919Z	INFO	[crawler]	beater/crawler.go:108	Loading and starting Inputs completed. Enabled inputs: 1
		2021-11-15 09:55:54	
2021-11-15T07:55:54.919Z	INFO	[crawler]	beater/crawler.go:141	Starting input (ID: 18216316265758894950)
2021-11-15 09:55:54	
2021-11-15T07:55:54.919Z	INFO	[aws-cloudwatch]	awscloudwatch/input.go:127	Initialized AWS CloudWatch input.
2021-11-15 09:55:54	
2021-11-15T07:55:54.918Z	WARN	[cfgwarn]	awscloudwatch/input.go:81	BETA: aws-clouwatch input type is used

I've tried building filebeat locally and added some logging and the prefix itself works i.e. logGroupNames in the input does contain all the log groups in my account, but filebeat only starts worker for one.

According to this Cherry-pick #26187 to 7.x: Add log_group_name_prefix config option for aws-cloudwatch input by kaiyan-sheng · Pull Request #26527 · elastic/beats · GitHub my config should work, is this a bug or is there something wrong with the filebeat.yml?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.