# S3 input assume role bug

**URL:** <https://discuss.elastic.co/t/s3-input-assume-role-bug/151606>\
**Category:** Logstash\
**Created:** [October 9, 2018, 10:06am UTC](https://discuss.elastic.co/t/s3-input-assume-role-bug/151606 "2018-10-09T10:06:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![shed7](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@shed7](https://discuss.elastic.co/u/shed7)\
**Post date:** [October 9, 2018, 10:06am UTC](https://discuss.elastic.co/t/s3-input-assume-role-bug/151606/1 "2018-10-09T10:06:20Z")

</div>

Hi!

I have an ec2 instance in eu-west-1a, an IAM role attached and I can see the temporary creds if I do:  
curl [http://169.254.169.254/latest/meta-data/iam/security-credentials/](http://169.254.169.254/latest/meta-data/iam/security-credentials/)

I have a small Python program to test if the temporary creds work, which looks like this:

```
    import boto3
import botocore

BUCKET_NAME = 'bucket-name' # replace with your bucket name
KEY = 'AWSLogs/TestFile' # replace with your object key

s3 = boto3.resource('s3')

try:
    s3.Bucket(BUCKET_NAME).download_file(KEY, 'TestFile')
except botocore.exceptions.ClientError as e:
    if e.response['Error']['Code'] == "404":
        print("The object does not exist.")
    else:
        raise

```

Running the above script in the ec2 instance works fine, it's definitely using the temporary creds as there are no environment variable set nor any .aws/credentials set either.  
I have also tested that the IAM role can access the bucket using the policy simulator, which it can.

However using the s3 input plugin I get Access Denied errors. Below is my config and then the errors:

```
input {
s3 {
    bucket => "bucket-name"
    region => "eu-central-1"
    role_arn => "full arn of the IAM role attached to this instance"
    role_session_name => "mysession"
    sincedb_path => "/var/lib/logstash/sincedb"
    type => "s3"
}

```

}

`[2018-10-09T11:04:36,568][ERROR][logstash.pipeline] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::S3 bucket=>\"bucket-name\", role_arn=>\"arn:aws:iam::999999999:role/rolename\", role_session_name=>\"mysession\", id=>\"eaaa92276690fd9e6d3fd8bb421e70670ebc78afe4010fd110f0c62bd6b54de2\", region=>\"eu-central-1\", sincedb_path=>\"/var/lib/logstash/sincedb\", type=>\"s3\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_ce7e1261-1b24-44c9-9b95-fe4d8df9d50f\", enable_metric=>true, charset=>\"UTF-8\">, delete=>false, interval=>60, temporary_directory=>\"/tmp/logstash\", include_object_properties=>false>", :error=>"Access denied", :thread=>"#<Thread:0x691ea2a9 run>"}`

[2018-10-09T11:04:38,062][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<Aws::STS::Errors::AccessDenied: Access denied\>, :backtrace=\>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/seahorse/client/plugins/raise\_response\_errors.rb:15:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in`call'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/plugins/idempotency\_token.rb:18:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/plugins/param_converter.rb:20:in`call'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/seahorse/client/plugins/response\_target.rb:21:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/seahorse/client/request.rb:70:in`send\_request'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/seahorse/client/base.rb:207:in `block in define_operation_methods'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/assume_role_credentials.rb:49:in`refresh'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/refreshing\_credentials.rb:20:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.104/lib/aws-sdk-core/assume_role_credentials.rb:40:in`initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-mixin-aws-4.3.0/lib/logstash/plugin\_mixins/aws\_config/v2.rb:69:in `assume_role'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-mixin-aws-4.3.0/lib/logstash/plugin_mixins/aws_config/v2.rb:63:in`credentials'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-mixin-aws-4.3.0/lib/logstash/plugin\_mixins/aws\_config/v2.rb:18:in `aws_options_hash'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:423:in`get\_s3object'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:86:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:241:in`register\_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in`each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:395:in`start\_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:293:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:199:in`run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:159:in `block in start'"], :thread=\>"#\<Thread:0x691ea2a9 run\>"}  
[2018-10-09T11:04:38,077][ERROR][logstash.agent] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Could not execute action: PipelineAction::Create, action\_result: false", :backtrace=\>nil}

Could anyone suggest a way of determining what the input plugin is actually doing, what creds it's trying to use, to try and get to the bottom of the problem.

Many thanks

---

<div class="post-metadata">

**Author:** ![shed7](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@shed7](https://discuss.elastic.co/u/shed7)\
**Post date:** [October 9, 2018, 11:28am UTC](https://discuss.elastic.co/t/s3-input-assume-role-bug/151606/2 "2018-10-09T11:28:19Z")

</div>

Just figured it out, don't specify the role\_arn, logstash will just pick up the temporary credentials from the ec2 instance metadata.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 6, 2018, 11:36am UTC](https://discuss.elastic.co/t/s3-input-assume-role-bug/151606/3 "2018-11-06T11:36:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.