Multipile s3 plugin inputs with different prefix

Elastic Stack Logstash
1

Hey!

I just wondering if it's a bug or feature.
my different ELBs are put logs to the same bucket but creates different prefix like:

s3://prod-elb-logs/[elb_name]/...

I use multiple s3 inputs with the same bucket but different prefix and only one input works:

input {
s3 {
bucket => "albs-dev-logs"
prefix => "elb1/AWSLogs/**********/elasticloadbalancing/us-east-1/"
region => "us-east-1"
codec => plain
type => "alb1_logs"
}
s3 {
bucket => "albs-dev-logs"
prefix => "elb2/AWSLogs/**************/elasticloadbalancing/us-east-1/"
region => "us-east-1"
codec => plain
type => "alb2_logs"
}
}

output {
if [type] == "alb1_logs" {
stdout { codec => rubydebug }
elasticsearch {
hosts => "es-logs-dev.es.com:80"
index => "alb1_logs-%{+YYYY.MM.dd}"
manage_template => true
template => "/etc/logstash/alb1_template.json"
template_overwrite => true
template_name => "alb_logs"
}
} else if [type] == "alb2_logs" {
stdout { codec => rubydebug }
elasticsearch {
hosts => "es-logs-dev.es.com:80"
index => "alb2_logs-%{+YYYY.MM.dd}"
manage_template => true
template => "/etc/logstash/alb2_template.json"
template_overwrite => true
template_name => "alb_logs"
}
}
}

any suggestions?

thanks!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.