Checking the S3 Input plugin documentation
This plugin uses the AWS SDK and supports several ways to get credentials, which will be tried in this order:
Static configuration, using access_key_id and secret_access_key params in logstash plugin config
External credentials file specified by aws_credentials_file
Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Environment variables AMAZON_ACCESS_KEY_ID and AMAZON_SECRET_ACCESS_KEY
IAM Instance Profile (available when running inside EC2)
The EC2 instance where Logstash is running currently has an Instance Profile set up that allows the access to read files from an S3 bucket. I also have a role that allows the instance to assume it and read the S3 bucket contents.
The problem is that if I configure the S3 input plugin to rely on it instead of user credentials, I get the following error:
S3 INPUT PLUGIN SETUP
input {
s3 {
id => "s3-input-cloudtrail"
bucket => "mycompany-cloudtrail-logs"
prefix => "AWSLogs/companyawsid/CloudTrail/"
codec => "json"
# With or without the next line commented, I get the same error message
#role_arn => "arn:aws:iam::companyawsid:role/LogstashS3ReadAccess"
}
}
ERROR
[2020-12-29T16:21:24,640][ERROR][logstash.javapipeline ][aws-cloudtrail][s3-input-cloudtrail] A plugin had an unrecoverable error. Wil
l restart this plugin.
Pipeline_id:aws-cloudtrail
Plugin: <LogStash::Inputs::S3 bucket=>"mycompany-cloudtrail-logs", codec=><LogStash::Codecs::JSON id=>"json_7c4671e2-8cea-49f7-a8a9-08
b0bbe0f099", enable_metric=>true, charset=>"UTF-8">, id=>"s3-input-cloudtrail", prefix=>"AWSLogs/companyawsid/CloudTrail/", enable_metric
=>true, region=>"us-east-1", role_session_name=>"logstash", delete=>false, interval=>60, watch_for_new_files=>true, temporary_directory=>
"/tmp/logstash", include_object_properties=>false, gzip_pattern=>".gz(ip)?$">
Error: unable to sign request without credentials set
Exception: Aws::Errors::MissingCredentialsError
Stack: /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/request_signer.rb:104:in `requ
ire_credentials'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_request_signer.rb:14:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_host_id.rb:14:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/xml/error_handler.rb:8:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_request_signer.rb:65:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_redirects.rb:15:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/retry_errors.rb:108:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_dualstack.rb:32:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_accelerate.rb:49:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_md5s.rb:31:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_iad_regional_endpoint.rb:31:in `call
'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_expect_100_continue.rb:21:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_bucket_name_restrictions.rb:12:in `c
all'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_bucket_dns.rb:31:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/rest/handler.rb:7:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/user_agent.rb:12:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/endpoint_pattern.rb:27:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/endpoint_discovery.rb:67:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/seahorse/client/plugins/endpoint.rb:41:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/param_validator.rb:21:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call
'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_dualstack.rb:24:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/s3_accelerate.rb:34:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/idempotency_token.rb:18:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/aws-sdk-core/plugins/response_paging.rb:26:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/seahorse/client/plugins/response_target.rb:21:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/seahorse/client/request.rb:70:in `send_request'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-core-2.11.605/lib/seahorse/client/base.rb:207:in `block in define_operation_me
thods'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-resources-2.11.605/lib/aws-sdk-resources/request.rb:24:in `call'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-resources-2.11.605/lib/aws-sdk-resources/operations.rb:139:in `all_batches'
org/jruby/RubyEnumerator.java:396:in `each'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/aws-sdk-resources-2.11.605/lib/aws-sdk-resources/collection.rb:18:in `each'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-3.5.0/lib/logstash/inputs/s3.rb:132:in `list_new_files'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-3.5.0/lib/logstash/inputs/s3.rb:172:in `process_files'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-3.5.0/lib/logstash/inputs/s3.rb:123:in `block in run'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/stud-0.0.23/lib/stud/interval.rb:20:in `interval'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-s3-3.5.0/lib/logstash/inputs/s3.rb:122:in `run'
/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:405:in `inputworker'
/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:396:in `block in start_input'
What might be the cause of this issue?