Is there any way to access all cloudwatch log groups without providing the exact name?

Elastic Stack Logstash
1

Hi everyone,

question about Logstash input plugin for Cloudwatch logs. I want to get all the logs in the log groups that start with /aws/lambda/, is that possible?

I tried the below configuration with log_group => ["/aws/lambda/*"], but it didn't work. However, using a specific log_group name works, but that is not what I intend to do.

Configuration:

input{
 cloudwatch_logs {
   access_key_id => "***"
   secret_access_key => "***"
   log_group => ["/aws/lambda/*"]
   region => "us-west-2"
 }
}

filter { ... }

output { ... }
2 
log_group_prefix => true

perhaps? I have not tested it, but from the code it appears to be what you want.

1 Like
3

Thanks for your suggestion, do I have to still specify the name of the log groups? or can I just use ["/aws/lambda/*"] with the * at the end of /aws/lambda/?

4

You can walk through the code and API, or you can experiment. My best guess is

log_group => "/aws/lambda/"
1 Like
5

@Badger, yes the log group prefix worked. Thank you! :grinning:

Solution:

input{
  cloudwatch_logs {
    access_key_id => "***"
    secret_access_key => "***"
    log_group_prefix => true
    log_group => ["/aws/lambda/"]
    region => "us-west-2"
  }
}
1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.