Comparing number in conditionnal event treatement

Elastic Stack Logstash
1

Hi there,

I have a problem with conditionnals events, when I use some of equalities (< ; > ; <= ; >= ) on a number field, I got a java error and logstash stop

Tried this :

input {
    beats
        { port => 5045 }
}

 filter {
    if [http][response][body][bytes] >= 2500
        {
         mutate 
            { 
            add_field => { "[test]" => "Body too long" }
            }
        }  }

[http][response][body][bytes] is of type 'long' in ES mapping.

Below error I've got.

2020-01-29T17:21:51,792][ERROR][org.logstash.execution.WorkerLoop][ws_v1-packetbeat] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter config
uration and restart Logstash.
java.lang.NullPointerException: null
        at org.logstash.config.ir.compiler.EventCondition$Compiler$UnexpectedTypeException.<init>(EventCondition.java:651) ~[logstash-core.jar:?]
        at org.logstash.config.ir.compiler.EventCondition$Compiler.compare(EventCondition.java:431) ~[logstash-core.jar:?]
        at org.logstash.config.ir.compiler.EventCondition$Compiler.lambda$compareFieldToConstant$11(EventCondition.java:422) ~[logstash-core.jar:?]
        at org.logstash.config.ir.compiler.Utils.filterEvents(Utils.java:27) ~[logstash-core.jar:?]
        at org.logstash.generated.CompiledDataset15.compute(Unknown Source) ~[?:?]
        at org.logstash.generated.CompiledDataset16.compute(Unknown Source) ~[?:?]
        at org.logstash.generated.CompiledDataset17.compute(Unknown Source) ~[?:?]
        at org.logstash.generated.CompiledDataset18.compute(Unknown Source) ~[?:?]
        at org.logstash.execution.WorkerLoop.run(WorkerLoop.java:64) [logstash-core.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_212]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_212]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]
        at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:440) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:304) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:36) [jruby-complete-9.2.8.0.jar:?]
        at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_workers$2(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:243) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.runtime.Block.call(Block.java:129) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.RubyProc.call(RubyProc.java:295) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.RubyProc.call(RubyProc.java:274) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.RubyProc.call(RubyProc.java:270) [jruby-complete-9.2.8.0.jar:?]
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105) [jruby-complete-9.2.8.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
[2020-01-29T17:21:51,887][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>java.lang.IllegalStateException: java.lang.NullPointerException, :backtrace=>["org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:440)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:304)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:243)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)", "java.lang.Thread.run(java/lang/Thread.java:748)"]}
[2020-01-29T17:21:52,002][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

If I do a comparison with == it works fine

if [http][response][body][bytes] == 2500 --> field is added

What am I doing wrong ?

2

Hi, any idea about that ?

3

If you use

output { stdout { codec => rubydebug } }

then is [http][response][body][bytes] shown as

2500

or

"2500"

?

4

[http][response][body][bytes] shown as number :

"http" => {
"response" => {
"headers" => {
"date" => "Mon, 02 Mar 2020 14:50:59 GMT",
"server" => "TwistedWeb/12.0.0",
"content-length" => 291,
"content-type" => "text/xml; charset="UTF-8"",
"connection" => "close"
},
"status_phrase" => "ok",
"status_code" => 200,
"bytes" => 455,
"body" => {
"bytes" => 291,
"content" => "Field deleted to save space"
}
}
},

5

That exception will occur if the field does not exist. For example...

input { generator { count => 1 lines => [ '' ] } }
filter {
    mutate { add_field => { "[foo]" => 2501 } }
    mutate { convert => { "[foo]" => integer } }

    if [bar] >= 2500 {
        mutate { add_field => { "[test]" => "Body too long" } }
    }
}
output { stdout { codec => rubydebug { metadata => false } } }

will produce that exception.

2 Likes
6

Thanks for the track, I figured out what was happening.

I a few documents, packetbeat generates 'unmatched responses', and in this very particular case [http][response][body][bytes] does not exists, and so logstash fails.

I added an exists test in my pipeline and now it works with no error

if ([http][response][body][bytes]) and [http][response][body][bytes] >= 2500

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.