Randomly crashes due to a TypeError - no implicit conversion of nil into Integer

Elastic Stack Logstash
1

Hello

I am using logstash 7.3.0, which is configured to read input from Kafka and output to elasticsearch or a different Kafka topic in certain cases. My config is as follows:

    input {
      kafka {
        bootstrap_servers => "...."
        client_id         => "logstash-1"
        auto_offset_reset => "earliest"
        consumer_threads  => 3
        topics            => ["logs"]
        codec             => "json"
      }
    }

    filter {
      date {
        match => [ "timestamp", "UNIX" ]
        remove_field => [ "timestamp", "@version" ]
      }
      if (![domain]) {
    	mutate {
    	    copy => { "email" => "temp" }
    	}
    	mutate {
    	    split => { "temp" => "@" }
    	}
    	mutate {
    	    add_field => {"domain" => "%{[temp][1]}"}
    	    remove_field => ["temp"]
    	}
      }
    }

    output {
      if (!([category]) or ([category] and [category] > 700)) {
        kafka {
          codec => "json"
          topic_id => "logs-1"
          bootstrap_servers => "...."
        }
      } else {
        elasticsearch {
          ilm_enabled => false
          user => "..."
      	  password => "..."
    	  manage_template => false
          hosts => [ 
            "..."
          ]    
          index => "logs"
        }
      }
    }

It randomly stops working due to a "TypeError - no implicit conversion of nil into Integer" error. I tried troubleshooting each part to reproduce the problem, but I can't seem to do so. It seems that no part of this configuration would crash in case of a nil value like the error says.

I would appreciate any advice. Thanks!

Below is the full error output:

[2020-03-27T10:34:51,966][ERROR][org.logstash.execution.WorkerLoop] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.
org.jruby.exceptions.TypeError: (TypeError) no implicit conversion of nil into Integer
	at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:239) ~[?:?]
warning: thread "[main]>worker7" terminated with exception (report_on_exception is true):
java.lang.IllegalStateException: org.jruby.exceptions.TypeError: (TypeError) no implicit conversion of nil into Integer
	at org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:440)
	at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:304)
	at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:239)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
	at java.lang.Thread.run(java/lang/Thread.java:834)
Caused by: org.jruby.exceptions.TypeError: (TypeError) no implicit conversion of nil into Integer
	at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:239)
[2020-03-27T10:34:52,141][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>java.lang.IllegalStateException: org.jruby.exceptions.TypeError: (TypeError) no implicit conversion of nil into Integer, :backtrace=>["org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:85)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)", "jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)", "jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)", "java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)", "org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:440)", "org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:304)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:239)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)", "org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)", "java.lang.Thread.run(java/lang/Thread.java:834)"]}
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.