[Filebeat] Bug with aws-cloudwatch logs using log_group_arn

Elastic Stack Beats
1

Hi! I have a filebeat system with the following configuration as an input:

filebeat.inputs:
    - type: aws-cloudwatch
      log_group_arn: arn:aws:logs:eu-west-1:*:log-group:/ecs/log:*
      scan_frequency: 30s
      start_position: beginning
      access_key_id: *
      secret_access_key: *

What I expected to happen is to get all the streams from the log group and output them to logstash. Instead the filebeat is not getting any logs and changes between aws-cloudwatch input worker has started and aws-cloudwatch input worker has stopped.
But when I change the configuration to:

filebeat.inputs:
    - type: aws-cloudwatch
      log_group_arn: arn:aws:logs:eu-west-1:*:log-group:/ecs/log:*
      log_streams: name_of_stream
      scan_frequency: 30s
      start_position: beginning
      access_key_id: *
      secret_access_key: *

It works only for the specified stream. I want to be able to get all the logs streams at the start of the filebeat execution and the real-time logs that will be created after the execution.

Thank you for your time!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.