Hi
I'm trying to get the S3 input plugin to work but it just fails saying it has an unrecoverable error. Hope that any of you recognize the error and can point me in the right direction because my googling is coming up short.
I've tried to see if it's the S3 policy that is wrong and used Error: uninitialized constant Aws::Client::Errors as input. But I can download a file via aws cli but the plugin still has errors. I've also tested on 2 different logstash setups, one in a docker container on a EC2 in aws and one locally on a ubuntu host, but got the same error on both setups.
The Logstash version is 5.2.0 and this is my input config
s3 {
bucket => "mybucket"
access_key_id => "access_key"
secret_access_key => "secret_key"
region => "eu-west-1"
delete => false
interval => 60 # seconds
type => "cloudtrail"
codec => "cloudtrail"
prefix => "AWSLogs/accountnr/CloudTrail/"
#sincedb_path => "/usr/share/logstash/data/cloudtrail_s3.sincedb"
}
The error I get is this:
12:51:17.589 [[main]<s3] ERROR logstash.pipeline - A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::S3 bucket=>"mybucket", access_key_id=>"access_key", secret_access_key=>"secret_key", region=>"eu-west-1", delete=>false, interval=>60, type=>"cloudtrail", codec=><LogStash::Codecs::Plain id=>"plain_588576c4-c58d-4b93-8e30-44e3bbb753c7", enable_metric=>true, charset=>"UTF-8">, prefix=>"AWSLogs/accountnr/CloudTrail/", id=>"id_nr", enable_metric=>true, temporary_directory=>"/tmp/logstash">
Error: execution expired
Exception: Seahorse::Client::NetworkingError
Stack: org/jruby/ext/socket/RubyTCPSocket.java:111:in initialize' org/jruby/RubyIO.java:1197:inopen'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in connect' org/jruby/ext/timeout/Timeout.java:115:intimeout'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:763:in connect' /usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:756:indo_start'
/usr/share/logstash/vendor/jruby/lib/ruby/1.9/net/http.rb:751:in start' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/net_http/connection_pool.rb:281:instart_session'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/net_http/connection_pool.rb:93:in session_for' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/net_http/handler.rb:116:insession'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/net_http/handler.rb:68:in transmit' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/net_http/handler.rb:42:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/plugins/content_length.rb:12:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_request_signer.rb:88:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_request_signer.rb:23:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/xml/error_handler.rb:8:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_request_signer.rb:65:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_redirects.rb:15:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:87:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:118:inretry_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:101:in retry_if_possible' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:89:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:118:in retry_request' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:101:inretry_if_possible'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:89:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:118:inretry_request'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:101:in retry_if_possible' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/retry_errors.rb:89:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_accelerate.rb:42:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_md5s.rb:31:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_expect_100_continue.rb:21:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_bucket_name_restrictions.rb:12:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_bucket_dns.rb:31:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/rest/handler.rb:7:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/user_agent.rb:12:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/plugins/endpoint.rb:41:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_url_encoded_keys.rb:39:in manage_keys' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_url_encoded_keys.rb:31:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/param_validator.rb:21:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/plugins/raise_response_errors.rb:14:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/s3_accelerate.rb:33:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/param_converter.rb:20:in call' /usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/aws-sdk-core/plugins/response_paging.rb:26:incall'
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/aws-sdk-core-2.3.22/lib/seahorse/client/plugins/response_target.rb:21:in `call'