[Filebeat] Bug with aws-cloudwatch logs using log_group_arn

Hi! I have a filebeat system with the following configuration as an input:

filebeat.inputs:
    - type: aws-cloudwatch
      log_group_arn: arn:aws:logs:eu-west-1:*:log-group:/ecs/log:*
      scan_frequency: 30s
      start_position: beginning
      access_key_id: *
      secret_access_key: *

What I expected to happen is to get all the streams from the log group and output them to logstash. Instead the filebeat is not getting any logs and changes between aws-cloudwatch input worker has started and aws-cloudwatch input worker has stopped.
But when I change the configuration to:

filebeat.inputs:
    - type: aws-cloudwatch
      log_group_arn: arn:aws:logs:eu-west-1:*:log-group:/ecs/log:*
      log_streams: name_of_stream
      scan_frequency: 30s
      start_position: beginning
      access_key_id: *
      secret_access_key: *

It works only for the specified stream. I want to be able to get all the logs streams at the start of the filebeat execution and the real-time logs that will be created after the execution.

Thank you for your time!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.