Also, just a heads up, Resource section in CFN should be as restrictive as possible. So instead of * give it the buckets that you know need the s3:PutObject permission.
That's the exact problem. Specifying a bucket ARN in Resources does not work. I tested the IAM policy limiting access to the specific bucket using the AWS CLI, and it worked as expected. However, it did not work with logstash.
In addition, I was only able to make this work with a bucket in the us-east-1 region (???).
@mujtabahussain I challenge you! Setup Logstash s3 output with logstash-5.4.3-1 to a bucket in us-west-2 region with a IAM user that is appropriately scoped to a single bucket. It will not work. And if it does work, I beg you to show me your configuration.
I've updated my post. I apologize for my temporary insanity. But it did seem that I had a mysterious problem with the s3 output plugin when I really had an unknown VPC Routing Endpoint policy preventing me from performing a PutObject on my bucket.
Thanks for the responses @mujtabahussain. I have it working now.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.