Logstash S3 input issues

Elastic Stack Logstash
1

Hi there,

I am having trouble trying to get Logstash to use the S3 input plugin to load log files from an S3 bucket.

This is the error I get:

[INFO ][logstash.inputs.s3 ] S3 input: No files found in bucket {:prefix=>"/test-recursion/*"}

Logstash version: 6.7

and this is my pipeline config:

input {
  beats {
    id => "beats-input"
    type => "k8-logs"
    port => 5044
  }
  s3 {
    id => "s3-input"
    type => "emr-logs"
    bucket => "MYBUCKETNAME"
    prefix => "/test-recursion/*"
    interval => 30
    region => "MYAWSREGION"
  }
}

I have tried all manner of * wildcards with no success of finding any files within any folder in a bucket.

HALP :0)

What am I missing?

Claire

OK so update. This plugin does seem to recursively look into buckets for logs but only if I remove the prefix setting.

Is that the expected behavior?

2

Hi friend, did you look into this thread ---> S3 Input not working, I guess this might help you out! :grinning:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.