How to use S3 Input Plugin with assume role_arn

Elastic Stack Logstash
1

Hi,

I am trying to find a way to use the S3 input plugin to ingest AWS Cloudtrail data. I have the config working but the way I am doing doesn't seem to be streamlined. So this is how I am able to make it work till now.

  1. I export the AWS Access Key, Secret Key, and Region as environment variables. Something like
    export AWS_ACCESS_KEY=xxxxxxxxxx
  2. Start logstash and specify the role_arn that has access to my S3 objects.

This is working.

But what I really want is a way to use a nicer approach to use the .aws/credentials file and specify the role_arn in my config to assume the role and the logstash should work.

But it seems like when I either use the access_key and secret_key parameter of the S3 input plugin along with role_arn. It doesn't seem to work. I believe that the input plugging is trying to access my bucket using my access keys instead of the role_arn. How can I achieve this?

Can someone help?

Also, can there be an option to specify Profiles. That would really be a very useful feature to the s3 input plugin. We use a lot of AWS CLI Profiles and having such a capability would be nice.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.