Is there any way to access all cloudwatch log groups without providing the exact name?

[EZprogramming (Kourosh Azizi)]

Hi everyone,

question about Logstash input plugin for Cloudwatch logs. I want to get all the logs in the log groups that start with /aws/lambda/, is that possible?

I tried the below configuration with log_group => ["/aws/lambda/*"], but it didn't work. However, using a specific log_group name works, but that is not what I intend to do.

Configuration:

input{
 cloudwatch_logs {
   access_key_id => "***"
   secret_access_key => "***"
   log_group => ["/aws/lambda/*"]
   region => "us-west-2"
 }
}

filter { ... }

output { ... }

[Badger]

log_group_prefix => true

perhaps? I have not tested it, but from the code it appears to be what you want.

[EZprogramming (Kourosh Azizi)]

@Badger

log_group_prefix => true

Thanks for your suggestion, do I have to still specify the name of the log groups? or can I just use ["/aws/lambda/*"] with the * at the end of /aws/lambda/?

[Badger]

@EZprogramming

Thanks for your suggestion, do I have to still specify the name of the log groups? or can I just use ["/aws/lambda/*"] with the * at the end of /aws/lambda/?

You can walk through the code and API, or you can experiment. My best guess is

log_group => "/aws/lambda/"

[EZprogramming (Kourosh Azizi)]

@Badger, yes the log group prefix worked. Thank you!

Solution:

input{
  cloudwatch_logs {
    access_key_id => "***"
    secret_access_key => "***"
    log_group_prefix => true
    log_group => ["/aws/lambda/"]
    region => "us-west-2"
  }
}