Hi there,
I am having trouble trying to get Logstash to use the S3 input plugin to load log files from an S3 bucket.
This is the error I get:
[INFO ][logstash.inputs.s3 ] S3 input: No files found in bucket {:prefix=>"/test-recursion/*"}
Logstash version: 6.7
and this is my pipeline config:
input {
beats {
id => "beats-input"
type => "k8-logs"
port => 5044
}
s3 {
id => "s3-input"
type => "emr-logs"
bucket => "MYBUCKETNAME"
prefix => "/test-recursion/*"
interval => 30
region => "MYAWSREGION"
}
}
I have tried all manner of * wildcards with no success of finding any files within any folder in a bucket.
HALP :0)
What am I missing?
Claire
OK so update. This plugin does seem to recursively look into buckets for logs but only if I remove the prefix setting.
Is that the expected behavior?
