Problem using external Ruby script in Logstash

Hello guys,

I'm having problem trying to get my ruby code filter into an external ruby file called test2.rb
When i was using a ruby {} inside my configuration, it was working very well, and i am new to ruby so maybe my mistakes are basics and i am sorry for this.

In my logstash.conf i'm calling ruby this way :

ruby {
		path => "C:\Users\T6SH\Desktop\Logstash\Ruby\test2.rb"
	}

Then my test2.rb looks like :

def filter(event)
	if event.get("[event][dataset]") == "system.process"
		if event.get("[process][name]").downcase == "svchost.exe"
			event.set("process.w3wp_application_pool", event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			event.set("process.app_name", "svchost - " + event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			return [event]
		elsif event.get("[process][name]").downcase == "java.exe"
			if event.get("[process][args]").include?("-jar")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
				return [event]				 
			elsif event.get("[process][args]").include?("-cp")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])	
				return [event]
			end
		else
			return event.set("process.app_name", event.get("[process][name]"))
		end
	elsif event.get("[event][dataset]") == "windows.service"
		if event.get("[windows][service][name]").downcase.start_with?("fgs","elasticsearch","w3svc","metricbeat","sos_scheduler_","msmq","msdtc")
		    return [event.set("windows.service.is_critical", true) ]
        else    
			return [event.set("windows.service.is_critical", false)   ]
		end
	end
end

My error :

Could not process event: Custom script did not return an array from 'filter'. Only arrays may be returned!

backtrace=>["C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:69:in `check_result_events!'", "C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:101:in `file_script'", 
"C:/Users/T6SH/Desktop/Logstash/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.5/lib/logstash/filters/ruby.rb:84:in `filter'", "C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:143:in `do_filter'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:162:in `block in multi_filter'", "org/jruby/RubyArray.java:1800:in `each'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/filters/base.rb:159:in `multi_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:115:in `multi_filter'", 
"C:/Users/T6SH/Desktop/Logstash/logstash-core/lib/logstash/java_pipeline.rb:243:in `block in start_workers'"]}

Before this i had a return for every event.set, but i didnt return array so i tried to had them in array then i tried to return like above an [event] array but i m losing my mind ...I feel like im coding blindly...

I hope i was clear enough, thanks for your future help and have a nice day !

Sincerly trying, Luigi

Hi again ,

I would like to add that my script is partially working... I mean when the process.name is svchost or java or not both, it runs the script i wanted, even if i stop all services and delete index patterns and datas... Then the part where event.dataset == windows.service is working too .... I must have a code that isnt working in all cases i dont know .. I hope this message will help you understand my problem ..

The script has to return an array of events. It is likely you will want all of them to be

return [event]

Hi @Badger,

Indeed i missed this one, sorry for this mistake. I re wrote my code a little bit to return only [event] array, but still got an error.

my file test2.rb :

def filter(event)
	if event.get("[event][dataset]") == "system.process"
		if event.get("[process][name]").downcase == "svchost.exe"
			event.set("process.w3wp_application_pool", event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			event.set("process.app_name", "svchost - " + event.get("[process][args]")[event.get("[process][args]").index("-k")+1])
			return [event]
		elsif event.get("[process][name]").downcase == "java.exe"
			if event.get("[process][args]").include?("-jar")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-jar")+1])
				return [event]				 
			elsif event.get("[process][args]").include?("-cp")
			    event.set("process.java_app_name", event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])
			    event.set("process.app_name", "java - " + event.get("[process][args]")[event.get("[process][args]").index("-cp")+2])	
				return [event]
			end
		else
			event.set("process.app_name", event.get("[process][name]"))
			return [event]
		end
	elsif event.get("[event][dataset]") == "windows.service"
		if event.get("[windows][service][name]").downcase.start_with?("fgs","elasticsearch","w3svc","metricbeat","sos_scheduler_","msmq","msdtc")
		    event.set("windows.service.is_critical", true) 
			return [event]
        else    
			event.set("windows.service.is_critical", false)
			return [event]
		end
	end
end

and the error is still the same :

 Could not process event: Custom script did not return an array from 'filter'. Only arrays may be returned!
backtrace =>... `check_result_events!' 
backtrace =>...  `file_script' .... backtrace =>...  `filter'
backtrace =>...  `do_filter' ..... backtrace =>...  `block in multi_filter'
backtrace =>...  'each'  .....  backtrace =>...  `multi_filter'
backtrace =>...  `block in start_workers'

Hi

You seem to be returning the event in all cases, except when [event][dataset] is neither system.process nor windows.service. If that case is a possibility, you might want to add an else to the main if in order to return [event] there as well.

The same for your [process][args] when it is neither -jar nor -cp.

Just a thought.

Hi @ITIC, thanks for your message.

I had the same idea so i tried to put it to handle all cases but nothing changed.. Then i saw that i wrote it bad (i wrote ' return [event} ')
So thanks again to remind me of testing a second time this solution. For all event from the system module that are not process i had to put this !

Have a great day !