Ruby filter help

Elastic Stack Logstash
1

Hello guys,

I've been struggling a lot with trying to put script in a logstash's pipeline. This script works on windows but not in CentOs 6.9. When I start logstash I have this error:

An unexpected error occurred! {:error=>#<NoMethodError: undefined method<' for nil:NilClass>`

I tried kinda everything. I don't know what to do. This is my script:

input {
  udp {
   	port => 5514
   	type => "syslog"
	} 
}
filter {
    grok {
      match => { "message" => "<%{NUMBER:sev}>%{GREEDYDATA:kvlist}" }
    }
    kv {
      source => "kvlist"
      remove_field => ["kvlist"]
    }
    ruby{
      code => 'category = ["0 Kern",
                "1 user",
                "2 mail",
                "3 daemon",
                "4 auth",
                "5 syslog",
                "6 lpr",
                "7 news",
                "8 uucp",
                "9 clock daemon",
                "10 authpriv",
                "11 FTP",
                "12 NTP system",
                "13 log audit",
                "14 log alert",
                "15 cron",
                "16 local0",
                "17 local1",
                "18 local2",
                "19 local3",
                "20 local4",
                "21 local5",
                "22 local6",
                "23 local7"]
gravity = [
                  "0 Emergency",
                  "1 Alert",
                  "2 Critical",
                  "3 Error",
                  "4 Warning",
                  "5 Notice",
                  "6 Informational",
                  "7 Debugging"]
$log = (event.get("sev") || "")
$temp = $log.to_i
$i = $temp % 8
$y = Integer($i/8)
$message = (category[$i].to_s ||"") 
$message += " "
$message += (gravity[$y].to_s || "")
puts("log description", $message)'
    }
}
output {
      elasticsearch {
        	hosts => [ "localhost:9200" ]
   	     	index => "syslog-%{+YYYY.MM.dd}"
          }
      stdout { codec => rubydebug }
}
2

If you want the full logs:

[2018-03-05T09:52:04,400][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2018-03-05T09:52:04,475][INFO ][logstash.pipeline        ] Pipeline started succesfully {:pipeline_id=>"pipeline1", :thread=>"#<Thread:0x52b6854e@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 sleep>"}
[2018-03-05T09:52:04,480][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:5514"}
[2018-03-05T09:52:04,494][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["pipeline1"]}
[2018-03-05T09:52:04,503][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:5514", :receive_buffer_bytes=>"62464", :queue_size=>"2000"}
[2018-03-05T09:52:04,931][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<NoMethodError: undefined method `<' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:213:in `get_event_type'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:165:in `event_action_params'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:39:in `event_action_tuple'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `block in multi_receive'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-elasticsearch-9.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:34:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:49:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:479:in `block in output_batch'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:478:in `output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:430:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:385:in `block in start_workers'"]}
[2018-03-05T09:52:05,003][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (NoMethodError) undefined method `<' for nil:NilClass
[2018-03-05T09:52:05,003][WARN ][logstash.inputs.udp      ] UDP listener died {:exception=>java.nio.channels.ClosedSelectorException, :backtrace=>["sun.nio.ch.SelectorImpl.keys(SelectorImpl.java:68)", "org.jruby.util.io.SelectorPool.put(SelectorPool.java:88)", "org.jruby.util.io.SelectExecutor.selectEnd(SelectExecutor.java:59)", "org.jruby.util.io.SelectExecutor.go(SelectExecutor.java:44)", "org.jruby.RubyIO.select(RubyIO.java:3405)", "org.jruby.RubyIO$INVOKER$s$0$3$select.call(RubyIO$INVOKER$s$0$3$select.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:743)", "org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:77)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:83)", "org.jruby.ir.instructions.CallBase.interpret(CallBase.java:428)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:355)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.InterpreterEngine.interpret(InterpreterEngine.java:83)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:179)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:165)", "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:338)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:163)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:314)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:73)", "org.jruby.ir.interpreter.Interpreter.INTERPRET_BLOCK(Interpreter.java:132)", "org.jruby.runtime.MixedModeIRBlockBody.commonYieldPath(MixedModeIRBlockBody.java:148)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:73)", "org.jruby.runtime.Block.call(Block.java:124)", "org.jruby.RubyProc.call(RubyProc.java:289)", "org.jruby.RubyProc.call(RubyProc.java:246)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:104)", "java.lang.Thread.run(Thread.java:748)"]}
3

no where i can see closing ' for code block in ruby filter unless its a typo error while copy paste. please use debug to get detailed trace.

4

Yup, sorry, typo error. What do you mean by use the debug?

5

run logstash command with --debug appended at the end.
ex: bin/logstash -f "conf path" --debug

6

As I run in Centos version 6.9 I run logstash with:

sudo initctl start logstash

Then I don't have a usual logstash file, so all my logs are stored in a file I copy and paste here.

7

this is running logstash through services if i am not wrong. how u installed logstash? in general linux machines, logstash bin folder will be at "/usr/share/logstash" and conf files will be at "/etc/logstash/confd/". check where your logstash binaries exists and run through terminal.

8

I followed this guide:

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

I used yum, then I did that:

sudo initctl start logstash

My configuration files are stored in "/etc/logstash" and my logs in "/var/log/logstash/". So I don't have a "/usr/share/logstash" folder

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.