logstash
Version : 5.1.1
Plugins installed: []
JVM version:
1.8.0
OS version:
centos 7.3.1611
Description of the problem including expected versus actual behavior:
Steps to reproduce:
I'm using rabbitmq input plugin and routing some data to s3
when routing to s3 I use conditional to check value of a nested json field [json_data][data][internal_id]
the s3 output works, but the pipeline crashes on some data and logstash stops consuming
the stacktrace shows that it tries to parse the internal_id as a number when evaluating the conditional
internal_id is never a number.. so I don't know how to fix this..help!!
I tried many workarounds.. like trying to use mutate to convert this field to string
but same error is thrown
another workaround i tried was to use json filter to add a tag to use in the output section
I also output all data to elasticsearch which always parses this field as a string correctly
Provide logs (if relevant):
Dec 30 15:31:43 ip-172-20-71-11 logstash: Exception in thread "[main]>worker0" java.lang.NumberFormatException: For input string: "internal_id"
Dec 30 15:31:43 ip-172-20-71-11 logstash: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at java.lang.Integer.parseInt(Integer.java:580)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at java.lang.Integer.parseInt(Integer.java:615)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.logstash.Accessors.fetch(Accessors.java:130)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.logstash.Accessors.get(Accessors.java:20)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.logstash.Event.getUnconvertedField(Event.java:160)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_get_field(JrubyEventExtLibrary.java:113)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent$INVOKER$i$1$0$ruby_get_field.call(JrubyEventExtLibrary$RubyEvent$INVOKER$i$1$0$ruby_get_field.gen)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.ast.IfNode.interpret(IfNode.java:110)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.Block.call(Block.java:101)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.RubyProc.call(RubyProc.java:300)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.internal.runtime.methods.ProcMethod.call(ProcMethod.java:64)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
Dec 30 15:31:43 ip-172-20-71-11 logstash: at rubyjit.LogStash::Pipeline$$output_batch_f5191bb5ffe5a291eb68a6882a0d6e80233bed6c1442407170.block_1$RUBY$file(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:324)