I am trying to find a way to use the S3 input plugin to ingest AWS Cloudtrail data. I have the config working but the way I am doing doesn't seem to be streamlined. So this is how I am able to make it work till now.
- I export the AWS Access Key, Secret Key, and Region as environment variables. Something like
- Start logstash and specify the role_arn that has access to my S3 objects.
This is working.
But what I really want is a way to use a nicer approach to use the .aws/credentials file and specify the role_arn in my config to assume the role and the logstash should work.
But it seems like when I either use the access_key and secret_key parameter of the S3 input plugin along with role_arn. It doesn't seem to work. I believe that the input plugging is trying to access my bucket using my access keys instead of the role_arn. How can I achieve this?
Can someone help?
Also, can there be an option to specify Profiles. That would really be a very useful feature to the s3 input plugin. We use a lot of AWS CLI Profiles and having such a capability would be nice.