Pipeline aborted due to error - exception=>java.lang.ClassCastException - Error while fetching data using Salesforce plugin

Elastic Stack Logstash
1

Hi,

I am trying to pull the Salesforce data using the salesforce plugin available in Logstash. Below is my configuration.

ELK Version : 7.4

input {
  salesforce {
   use_test_sandbox => true
    client_id => 'xxx'
    client_secret => 'xxx'
    username => 'xxx'
    password => 'xxx'
    security_token => 'xxx'
    sfdc_object_name => 'xxx'
  }
}
output {
 elasticsearch {
    hosts => ["https://xxx:443"]   //443 because using Managed ElasticSearch
    index => "salesforcelog-%{+YYYY.MM}"
       user => "xxx"
        password => "xxx"
        ilm_enabled => false   //using this for Managed ES 
  }
    stdout { codec => rubydebug }
}

I end up getting the below error.

[ERROR] 2020-10-27 09:16:09.119 [[main]-pipeline-manager] javapipeline - Pipeline aborted due to error {:pipeline_id=>"main", :exception=>java.lang.ClassCastException: org.jruby.ext.set.RubySet cannot be cast to org.jruby.RubyFixnum

Please help me resolve this !! Thank you

2

Is there a stack trace for that error message?

3

Hi Thank you for the reply !!

Please find the complete message :

`[ERROR] 2020-10-27 16:33:28.392 [[main]-pipeline-manager] javapipeline - Pipeline aborted due to error {:pipeline_id=>"main", :exception=>java.lang.ClassCastException: org.jruby.ext.set.RubySet cannot be cast to org.jruby.RubyFixnum, :backtrace=>["org.jruby.runtime.invokedynamic.MathLinker.fixnum_op_lshift(MathLinker.java:297)", "java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)", "org.jruby.runtime.invokedynamic.MathLinker.fixnumOperator(MathLinker.java:171)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_middleware_minus_0_dot_14_dot_0.lib.faraday_middleware.response.follow_redirects.RUBY$method$initialize$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday_middleware-0.14.0/lib/faraday_middleware/response/follow_redirects.rb:71)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.runtime.IRRuntimeHelpers.classSuper(IRRuntimeHelpers.java:1171)", "org.jruby.ir.runtime.IRRuntimeHelpers.classSuperSplatArgs(IRRuntimeHelpers.java:1161)", "org.jruby.ir.targets.ClassSuperInvokeSite.invoke(ClassSuperInvokeSite.java:38)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.middleware.RUBY$method$new$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/middleware.rb:21)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$build$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:48)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$block$to_app$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:167)", "org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146)", "org.jruby.runtime.BlockBody.yield(BlockBody.java:122)", "org.jruby.runtime.Block.yieldArray(Block.java:182)", "org.jruby.RubyEnumerable$27.call(RubyEnumerable.java:1077)", "org.jruby.runtime.BlockCallback.call(BlockCallback.java:40)", "org.jruby.runtime.CallBlock.doYield(CallBlock.java:92)", "org.jruby.runtime.BlockBody.yield(BlockBody.java:116)", "org.jruby.runtime.Block.yield(Block.java:170)", "org.jruby.RubyArray.each(RubyArray.java:1800)", "org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)", "org.jruby.RubyClass.finvoke(RubyClass.java:493)", "org.jruby.runtime.Helpers.invoke(Helpers.java:430)", "org.jruby.RubyEnumerable.callEach(RubyEnumerable.java:103)", "org.jruby.RubyEnumerable.injectCommon(RubyEnumerable.java:1072)", "org.jruby.RubyEnumerable.inject(RubyEnumerable.java:1093)", "org.jruby.RubyEnumerable$INVOKER$s$inject.call(RubyEnumerable$INVOKER$s$inject.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrTwoBlock.call(JavaMethod.java:602)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:177)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$to_app$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:167)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$to_app$0$VARARGS(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$app$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:156)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$app$0$VARARGS(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$build_response$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb:143)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.rack_builder.RUBY$method$build_response$0$VARARGS(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/rack_builder.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.connection.RUBY$method$run_request$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:387)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.connection.RUBY$method$run_request$0$VARARGS(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:177)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.faraday_minus_0_dot_15_dot_4.lib.faraday.connection.RUBY$method$get$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/faraday-0.15.4/lib/faraday/connection.rb:138)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.RubyClass.finvoke(RubyClass.java:505)", "org.jruby.RubyBasicObject.send(RubyBasicObject.java:1741)", "org.jruby.RubyKernel.send(RubyKernel.java:2193)", "org.jruby.RubyKernel$INVOKER$s$send.call(RubyKernel$INVOKER$s$send.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.restforce_minus_4_dot_2_dot_2.lib.restforce.concerns.verbs.RUBY$block$define_verb$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:37)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.internal.runtime.methods.ProcMethod.call(ProcMethod.java:64)", "org.jruby.RubyClass.finvoke(RubyClass.java:505)", "org.jruby.RubyBasicObject.send(RubyBasicObject.java:1741)", "org.jruby.RubyKernel.send(RubyKernel.java:2193)", "org.jruby.RubyKernel$INVOKER$s$send.call(RubyKernel$INVOKER$s$send.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.restforce_minus_4_dot_2_dot_2.lib.restforce.concerns.verbs.RUBY$block$define_api_verb$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/verbs.rb:63)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.internal.runtime.methods.ProcMethod.call(ProcMethod.java:64)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.restforce_minus_4_dot_2_dot_2.lib.restforce.concerns.api.RUBY$method$describe$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/restforce-4.2.2/lib/restforce/concerns/api.rb:128)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_salesforce_minus_3_dot_0_dot_6.lib.logstash.inputs.salesforce.RUBY$method$register$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb:90)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_salesforce_minus_3_dot_0_dot_6.lib.logstash.inputs.salesforce.RUBY$method$register$0$VARARGS(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-salesforce-3.0.6/lib/logstash/inputs/salesforce.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$register_plugins$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195)", "org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146)", "org.jruby.runtime.BlockBody.yield(BlockBody.java:114)", "org.jruby.runtime.Block.yield(Block.java:170)", "org.jruby.RubyArray.each(RubyArray.java:1800)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$start_inputs$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:296)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$start_inputs$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$start_workers$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:252)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$start_workers$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$run$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$run$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)",

4 
"org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.RubyProc.call(RubyProc.java:274)", "org.jruby.RubyProc.call(RubyProc.java:270)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"], :thread=>"#<Thread:0x34c898b run>"}`
[ERROR] 2020-10-27 16:33:28.415 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[DEBUG] 2020-10-27 16:33:28.457 [Api Webserver] agent - Starting puma
[DEBUG] 2020-10-27 16:33:28.465 [Api Webserver] agent - Trying to start WebServer {:port=>9600}
[DEBUG] 2020-10-27 16:33:28.473 [LogStash::Runner] os - Stopping
[DEBUG] 2020-10-27 16:33:28.500 [LogStash::Runner] jvm - Stopping
[DEBUG] 2020-10-27 16:33:28.501 [LogStash::Runner] persistentqueue - Stopping
[DEBUG] 2020-10-27 16:33:28.502 [LogStash::Runner] deadletterqueue - Stopping
[DEBUG] 2020-10-27 16:33:28.506 [LogStash::Runner] agent - Shutting down all pipelines {:pipelines_count=>0}
[DEBUG] 2020-10-27 16:33:28.508 [LogStash::Runner] agent - Converging pipelines state {:actions_count=>0}
[DEBUG] 2020-10-27 16:33:28.521 [Api Webserver] service - [api-service] start
[INFO ] 2020-10-27 16:33:28.656 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2020-10-27 16:33:33.715 [LogStash::Runner] runner - Logstash shut down.
5

That is strange. The salesforce client is initializing, and during initialization it does a describe of @sfdc_object_name. I believe that involves doing a GET on it. Down inside the faraday HTTP client it appears to blow up at this line of code.

Looking at the error message

:exception=>java.lang.ClassCastException: org.jruby.ext.set.RubySet cannot be cast to org.jruby.RubyFixnum, 
:backtrace=>["org.jruby.runtime.invokedynamic.MathLinker.fixnum_op_lshift(MathLinker.java:297)",
"java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)", 
"org.jruby.runtime.invokedynamic.MathLinker.fixnumOperator(MathLinker.java:171)",

fixnum_op_lshift is what gets the exception. I cannot conceive how

  @convert_to_get = Set.new [303]
  @convert_to_get << 301 << 302 unless standards_compliant?

could throw that exception. @convert_to_get is a Set, why would it try to cast it to a Fixnum when adding a Fixnum (301) to the set?

This is in the initialize function, so I would expect it to always go through this code. It is not processing a redirect at this point, so it is not handling a response from Salesforce, it is initializing so that it can do so in the future.

I am baffled. Sorry I cannot be more help.

6

maybe just by keeping a xml file, your whole issue may get solved and i got a similar answer..this may help you i guess https://techanswered.com/question/e-java-lang-noclassdeffounderror-javax-xml-bind-jaxbexception/

7

Hi ,

Thank you for the replies.

It seemed more of an issue with a Logstash version i.e 7.4

I upgraded logstash alone to 7.9 and executed the same code for Salesforce and it worked and index got created in my Managed ES.