Problem using external Ruby script in Logstash

Elastic Stack Logstash
1

Hello guys,

I'm having problem trying to get my ruby code filter into an external ruby file called test2.rb
When i was using a ruby {} inside my configuration, it was working very well, and i am new to ruby so maybe my mistakes are basics and i am sorry for this.

In my logstash.conf i'm calling ruby this way :

ruby {
		path => "C:\Users\T6SH\Desktop\Logstash\Ruby\test2.rb"
	}

Then my test2.rb looks like :

def filter(event)
	if event.get("[event][dataset]") == "system.process"
		if event.get("[process][name]").downcase == "svchost.exe"
			event.set("process.w3wp_application_pool", event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			event.set("process.app_name", "svchost - " + event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			return [event]
		elsif event.get("[process][name]").downcase == "java.exe"
			if event.get("[process][args]").include?("-jar")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
				return [event]				 
			elsif event.get("[process][args]").include?("-cp")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])	
				return [event]
			end
		else
			return event.set("process.app_name", event.get("[process][name]"))
		end
	elsif event.get("[event][dataset]") == "windows.service"
		if event.get("[windows][service][name]").downcase.start_with?("fgs","elasticsearch","w3svc","metricbeat","sos_scheduler_","msmq","msdtc")
		    return [event.set("windows.service.is_critical", true) ]
        else    
			return [event.set("windows.service.is_critical", false)   ]
		end
	end
end

My error :

Could not process event: Custom script did not return an array from 'filter'. Only arrays may be returned!

backtrace=>["C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:69:in `check_result_events!'", "C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:101:in `file_script'", 
"C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:84:in `filter'", "C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:143:in `do_filter'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:162:in `block in multi_filter'", "org/jruby/RubyArray.java:1800:in `each'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:159:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:115:in `multi_filter'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/java_pipeline.rb:243:in `block in start_workers'"]}

Before this i had a return for every event.set, but i didnt return array so i tried to had them in array then i tried to return like above an [event] array but i m losing my mind ...I feel like im coding blindly...

I hope i was clear enough, thanks for your future help and have a nice day !

Sincerly trying, Luigi

2

Hi again ,

I would like to add that my script is partially working... I mean when the process.name is svchost or java or not both, it runs the script i wanted, even if i stop all services and delete index patterns and datas... Then the part where event.dataset == windows.service is working too .... I must have a code that isnt working in all cases i dont know .. I hope this message will help you understand my problem ..

3

The script has to return an array of events. It is likely you will want all of them to be

return [event]
4

Hi @Badger,

Indeed i missed this one, sorry for this mistake. I re wrote my code a little bit to return only [event] array, but still got an error.

my file test2.rb :

def filter(event)
	if event.get("[event][dataset]") == "system.process"
		if event.get("[process][name]").downcase == "svchost.exe"
			event.set("process.w3wp_application_pool", event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			event.set("process.app_name", "svchost - " + event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			return [event]
		elsif event.get("[process][name]").downcase == "java.exe"
			if event.get("[process][args]").include?("-jar")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
				return [event]				 
			elsif event.get("[process][args]").include?("-cp")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])	
				return [event]
			end
		else
			event.set("process.app_name", event.get("[process][name]"))
			return [event]
		end
	elsif event.get("[event][dataset]") == "windows.service"
		if event.get("[windows][service][name]").downcase.start_with?("fgs","elasticsearch","w3svc","metricbeat","sos_scheduler_","msmq","msdtc")
		    event.set("windows.service.is_critical", true) 
			return [event]
        else    
			event.set("windows.service.is_critical", false)
			return [event]
		end
	end
end

and the error is still the same :

 Could not process event: Custom script did not return an array from 'filter'. Only arrays may be returned!
backtrace =>... `check_result_events!' 
backtrace =>...  `file_script' .... backtrace =>...  `filter'
backtrace =>...  `do_filter' ..... backtrace =>...  `block in multi_filter'
backtrace =>...  'each'  .....  backtrace =>...  `multi_filter'
backtrace =>...  `block in start_workers'
5

Hi

You seem to be returning the event in all cases, except when [event][dataset] is neither system.process nor windows.service. If that case is a possibility, you might want to add an else to the main if in order to return [event] there as well.

The same for your [process][args] when it is neither -jar nor -cp.

Just a thought.

6

Hi @ITIC, thanks for your message.

I had the same idea so i tried to put it to handle all cases but nothing changed.. Then i saw that i wrote it bad (i wrote ' return [event} ')
So thanks again to remind me of testing a second time this solution. For all event from the system module that are not process i had to put this !

Have a great day !

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.