[ERROR][logstash.filters.ruby ] Ruby exception occur red: too few arguments : Need Help

Hi ,

I am getting Ruby Exception too few arguments everytime I run logstash on our server . Below is the config and error msg I am trying to Parse :slight_smile:
Message :
20200515T005009.898-0500 Warning Fail Connect to: ap45467 Error:No connection could be made because the target machine actively refused it [::ffff:10.178.8.144]:4747

Config :
######################
##Source: INPUT##
######################
input {
file {
path => "C:/ProgramData/QlikTech/WebServer/Log/*.txt"
start_position => "beginning"
ignore_older => 86400
#sincedb_path => "C:/ProgramData/QlikTech/WebServer/Log/20200604.txt"
}
}

####################################################
##Filter to mutate, format, convert to JSON##
####################################################

filter {

mutate {
gsub => ["message", "\n", "" ]
}
mutate
{
gsub => ["message", "\t", " " ]

}
#pattern matching for the log.
grok{
match=>{
"message"=>"%{GREEDYDATA:receivedTime}\t%{WORD:loglevel}\t%{GREEDYDATA:msg} %{GREEDYDATA:msg1} %{GREEDYDATA:notneeded}"
}
}
#set new field to receivedTime
mutate {
add_field => {
"receivedTime" => "%{@timestamp}"
}
}

	ruby {
	        init => "require 'date'"
	        code => "event.set('receivedTime', DateTime.parse(event.get('receivedTime')).strftime('%Q').to_i)"
	    }

#ruby {

code => "event.set('receivedTime',event.get('@timestamp').to_i * 1000)"

#}

#date plugin to convert timestamp => ISO8601

date {

match => ["receivedTime","YYYY-MM-dd HH:mm:ss", "ISO8601"]

target => "@timestamp"

}

#set field for hostname, socket to retrieve hostname
ruby {
init => "require 'socket'"
code => "event.set('hostname', Socket.gethostname)"
}

#set field for ipv4, socket for resolving hostname => ipv4
ruby {
init => "require 'socket'"
code => "event.set('source_ipv4', Socket.gethostname)"
}

#dns filter plugin to resolve hostname => ipv4
dns {
failed_cache_size => "2000"
failed_cache_ttl => "600"
hit_cache_size => "2000"
hit_cache_ttl => "600"
max_retries => "3"
resolve => [ "source_ipv4" ]
action => "replace"
timeout => 5
}

#convert ipv4 => ip long
ruby {
init => "require 'ipaddr'"
code => 'event.set("source_ipv4",event.get("source_ipv4").strip.split(".").map(&:to_i).pack("CCCC").unpack("N")[0])'
}

#mutate plugin to map column names => schema fields & implement nested JSON
mutate {

#application object with askId, name
add_field => {
  "[application][askId]" => "xyz"
  "[application][name]" => "8989"
   }
#device object with vendor, product, hostname, ip4
add_field => {
  "[device][vendor]" => "abc"
  "[device][product]" => "Ql"
  "[device][hostname]" => "%{[hostname]}.uhc.com"
  "[device][ip4]" => "%{source_ipv4}"
}
#required fields object with logClass
add_field => {
  "logClass" => "UNCATEGORIZED"
}

}
#convert ip4 => integer
mutate {
convert => {
"[device][ip4]" => "integer"
}
}

#if "_grokparsefailure" in [tags]{
#if "Authentication failed" in [message]{

ruby {

code => 'event.set("uid",event.get("message").strip.split("'")[1].strip.split(",")[0])'

}

 # mutate{
	#	 add_field =>{
	#	 "time_zone" => "CST"
	#	 "log_detail" => "Error"
		# "application_name" => "STARS"
		# "msg" => "Invalid Credentials %{[uid]}"
	#	 "date" => "%{+MM/dd/YY hh:mm:ss:ms}"
	#	 }

#	}

}

else{

drop {}

#}
#}
#remove all unnecessary fields
mutate {
remove_field => [ "notneeded", "num", "num2", "log_detail", "value", "log_class", "msg2", "date", "host", "@timestamp", "application_name", "log_detail", "source_ipv4", "hostname", "uid"]
}
}

#############################
##Send logs to Kafka stream##
#######################################################
##Dev topic name: com_eis_dasi_ness_producer_dev_core##
#######################################################
output {
if (("Error:" in [msg])){
stdout {codec => json}
file {
path => "C:/Users/1234/Downloads/QlikviewLogs/events.txt"
}
kafka {
bootstrap_servers => "apvrd21669.uhc.com:9093,apvrd21671.uhc.com:9093,apvrd22233.uhc.com:9093"
security_protocol => "SSL"
ssl_truststore_location => "C:/Users/1234/Downloads/certs/myTrustStore.jks"
ssl_truststore_password => "smrt2020"
topic_id => "com_eis_dasi_ness_producer_dev_core"
codec => "json"
}
}

}

What is the complete error message?

[2020-06-23T06:48:31,556][INFO ][logstash.agent ] Successfully started
Logstash API endpoint {:port=>9600}
[2020-06-23T06:48:35,596][ERROR][logstash.filters.ruby ] Ruby exception occur
red: too few arguments

Is there no stacktrace?

no , I am guessing some issue in my message parsing incase if \t in my grok pattern causing the exception ? Not sure though but its coming from filter block of code

The problem is not in the grok filter, it is in one of the ruby filters. I would try commenting them all out then adding them back one at a time to see which one results in an exception.

Ok will give it a try and let u know

started commenting out all ruby blocks one by one , got exception when this part of Ruby code was included :

#convert ipv4 => ip long
ruby {
init => "require 'ipaddr'"
code => 'event.set("source_ipv4",event.get("source_ipv4").strip.split(".").map(&:to_i).pack("CCCC").unpack("N")[0])'
}

FYI , I had not ot any execption in my local but I do get this exception on our stage servers . Kindly help as I am completely new to this area

What do you expect that to do?

not sure , we were just given a template to add our details for logging and compliance . Not sure what it does , it can be ignored if not needed.

Seems like our server is having ipV6 so its not working any replacement code for ipv6 will work

Code like

event.get("source_ipv4").strip.split(".").map { |x| x.to_i }.pack("CCCC").unpack("N")

will take apart the 4 octets of an IPv4 address and then pack them into a 32-bit number. IPv4 addresses are 32-bits, so this is a good fit.

IPv6 addresses are 128-bits. How do expect to represent that as a number?