S3 input "include pattern"?

[Allie_Yang (Allie Yang Yang)]

I have many files in my s3 bucket, is there a way to specify the pattern I want to include?

They are in several different sub-buckets, so i don't think i can use prefix..

[pierhugues (Pier-Hugues Pellerin)]

Hello,

The S3 inputs doesn't current support any option to only match only on specific pattern, but the plugin has an exclude _pattern, Could that work for your case?

It support Ruby style format, the following example will ignore any key in the bucket starting with "hello"

s3 {
#...
exclude_pattern => "^hello"
}

Ref: https://github.com/logstash-plugins/logstash-input-s3/blob/master/lib/logstash/inputs/s3.rb#L55