Can't understand an error from AWS S3 input in Logstash

KIVagant · October 1, 2018, 2:47pm

Hello.
I have a problem with the Logstash AWS S3 input plugin. I have several *.conf files in /etc/logstash/conf.d/ and can't even understand which of them is causing the error. When I run each of them separately with DEBUG mode all of them work fine. But when I start Logstash as service, I see the next trace in /var/log/logstash/logstash-plain.log

 [2018-10-01T14:39:18,518][ERROR][logstash.pipeline        ] ms/aws-sdk-core-2.11.127/lib/seahorse/client/http/response.rb:130)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.http.response.RUBY$method$on_success$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/http/response.rb:137)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.response.RUBY$method$on$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/response.rb:44)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.rest.handler.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/rest/handler.rb:8)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.user_agent.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/user_agent.rb:12)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.plugins.endpoint.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/plugins/endpoint.rb:41)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.s3_url_encoded_keys.RUBY$method$manage_keys$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/s3_url_encoded_keys.rb:40)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.s3_url_encoded_keys.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/s3_url_encoded_keys.rb:32)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.param_validator.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/param_validator.rb:21)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.plugins.raise_response_errors.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/plugins/raise_response_errors.rb:14)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.s3_sse_cpk.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.s3_dualstack.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/s3_dualstack.rb:24)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.s3_accelerate.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/s3_accelerate.rb:34)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.jsonvalue_converter.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20)

Please, help me to understand how to find the real error message in cases such this.

... see the second part of the trace in the comment to the topic.

KIVagant · October 1, 2018, 2:48pm

usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.idempotency_token.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/idempotency_token.rb:18)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.param_converter.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/param_converter.rb:20)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_core.plugins.response_paging.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/aws-sdk-core/plugins/response_paging.rb:26)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.plugins.response_target.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/plugins/response_target.rb:21)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.request.RUBY$method$send_request$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/request.rb:70)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_core_minus_2_dot_11_dot_127.lib.seahorse.client.base.RUBY$block$define_operation_methods$2(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-core-2.11.127/lib/seahorse/client/base.rb:207)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:145)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)
org.jruby.runtime.Block.call(Block.java:124)
org.jruby.RubyProc.call(RubyProc.java:289)
org.jruby.internal.runtime.methods.ProcMethod.call(ProcMethod.java:63)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:204)
org.jruby.RubyClass.finvoke(RubyClass.java:531)
org.jruby.RubyBasicObject.send19(RubyBasicObject.java:1670)
org.jruby.RubyKernel.send19(RubyKernel.java:1986)
org.jruby.RubyKernel$INVOKER$s$send19.call(RubyKernel$INVOKER$s$send19.gen)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_resources_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_resources.request.RUBY$method$call$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.127/lib/aws-sdk-resources/request.rb:24)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_resources_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_resources.operations.RUBY$method$all_batches$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.127/lib/aws-sdk-resources/operations.rb:139)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_resources_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_resources.operations.RUBY$method$all_batches$0$__VARARGS__(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.127/lib/aws-sdk-resources/operations.rb)
org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:77)
org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:93)
org.jruby.RubyClass.finvoke(RubyClass.java:522)
org.jruby.runtime.Helpers.invoke(Helpers.java:395)
org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:393)
org.jruby.RubyEnumerator.each(RubyEnumerator.java:323)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.aws_minus_sdk_minus_resources_minus_2_dot_11_dot_127.lib.aws_minus_sdk_minus_resources.collection.RUBY$method$each$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/aws-sdk-resources-2.11.127/lib/aws-sdk-resources/collection.rb:18)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_s3_minus_3_dot_3_dot_7.lib.logstash.inputs.s3.RUBY$method$list_new_files$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:119)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_s3_minus_3_dot_3_dot_7.lib.logstash.inputs.s3.RUBY$method$process_files$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:159)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_s3_minus_3_dot_3_dot_7.lib.logstash.inputs.s3.RUBY$block$run$1(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:110)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:145)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)
org.jruby.runtime.Block.call(Block.java:124)
org.jruby.RubyProc.call(RubyProc.java:289)
org.jruby.RubyProc.call19(RubyProc.java:273)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.stud_minus_0_dot_0_dot_23.lib.stud.interval.RUBY$method$interval$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:20)
usr.share.logstash.vendor.bundle.jruby.$2_dot_3_dot_0.gems.logstash_minus_input_minus_s3_minus_3_dot_3_dot_7.lib.logstash.inputs.s3.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-s3-3.3.7/lib/logstash/inputs/s3.rb:109)
usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:409)
usr.share.logstash.logstash_minus_core.lib.logstash.pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:403)
org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:145)
org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71)
org.jruby.runtime.Block.call(Block.java:124)
org.jruby.RubyProc.call(RubyProc.java:289)
org.jruby.RubyProc.call(RubyProc.java:246)
org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)
java.lang.Thread.run(Thread.java:748)

system · October 29, 2018, 2:48pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.