Hi All,
We have requirement which is little bit different to write configuration logstash.
Below are the files and location details
s3bucket/logs/prod/todate/prod.txt
s3bucket/logs/dev/todate/dev.txt
The input config has to read txt files from all environment directories and needs to insert in to respective indexes. Below is the one i wrote but still not clear
input {
s3 { type => "logs"
bucket => "s3bucket"
prefix => [ "logs/dev/", "logs/prod/" ]
interval => 20
access_key_id => "XXXXXXXXX"
secret_access_key => "XXXXXXXXXXXX"
}
}
As per logstash docs for s3 input prefix is not supported for regex operations but is there any other s3 input plugin available which will accept regex operations
like path => for file input
Here main agenda is read files from multiple directories with in s3 bucket and insert docs into respective index
Please provide suggestions to develop this input configuration
Thanks