So we are trying to use Logstash to send out some logs to s3 and its not working for some reason. This is the code we are using.
output {
amazon_es {
hosts => ["xxxx"]
region => "xxxx"
index => "%{[fields][log_type]}-%{+YYYY.MM.dd}"
}
s3 {
bucket => 'xxxx'
region => 'xxxx'
time_file => 5
role_arn => 'xxxx'
prefix => '%{+YYYY}/%{+MM}/%{+dd}/xxxx'
codec => 'json_lines'
}
}
Please note that this very same code worked in our non-prod environment, it is only troubling us in Production. Also, the amazon_es block is working fine in Production, its only the s3 part which isn't working. Its not an iam role issue, it has all the permissions possible assigned to it at this point and its still not working.
I have attached the error logs screenshot of when we try and start up logstash with the s3 output block. Would appreciate some insight on this. We can't really get the whole logs from production - but these are the only relevant logs for this however.
