Use script in Logstash

Elastic Stack Logstash
1

Hello,

I have a question, can we use a script in Logstash pipeline? I found that we can use ruby code in the pipeline but can we use another language or maybe execute the script file we want ?

This is the pipeline I want to use:

input {
  udp {
   	port => 514
   	type => "syslog"
	} 
}
filter {
    grok {
      match => { "message" => "<%{NUMBER:sev}>%{GREEDYDATA:kvlist}" }
    }
    kv {
      source => "kvlist"
      remove_field => ["kvlist"]
    }
}
output {
      elasticsearch {
        	hosts => [ "localhost:9200" ]
   	     	user => elastic
       		password => elasticlourd
          index => "syslog-%{+YYYY.MM.dd}"
          }
      #stdout { codec => rubydebug }
}

the "sev" field is the information I need for the script, can I give it to my script and then the script write the result in a variable created in logstash?

2

There's no generic "run script" filter but you can run a script from within a ruby filter.

3

Ok then, so if I need to get the field "sev" I created in the conf file in my ruby code, do I need to do this:
log = event.get("sev").value
and if I want to create a new field for the logs do I need to do this :
event.set("log description", var)
?
Regards.

4

No need for .value but otherwise correct.

5

I'll test that, thank you.

6

It's me again, I used this configuration of Logstash and it was working until now, I don't know what changed, but when I use the script in the conf file it doesn't sent logs to elasticsearch, but when I take it away, it works. With the plugin "ruby" in, it says an error like this
Ruby exception occurred: no implicit conversion of nil into String

my conf file:

input {
  udp {
   	port => 5514
   	type => "syslog"
	} 
}
filter {
    grok {
      match => { "message" => "<%{NUMBER:sev}>%{GREEDYDATA:kvlist}" }
    }
    kv {
      source => "kvlist"
      remove_field => ["kvlist"]
    }
    ruby{
      code => 'category = ["0 Kern",
                "1 user",
                "2 mail",
                "3 daemon",
                "4 auth",
                "5 syslog",
                "6 lpr",
                "7 news",
                "8 uucp",
                "9 clock daemon",
                "10 authpriv",
                "11 FTP",
                "12 NTP system",
                "13 log audit",
                "14 log alert",
                "15 cron",
                "16 local0",
                "17 local1",
                "18 local2",
                "19 local3",
                "20 local4",
                "21 local5",
                "22 local6",
                "23 local7"]
gravity = [
                  "0 Emergency",
                  "1 Alert",
                  "2 Critical",
                  "3 Error",
                  "4 Warning",
                  "5 Notice",
                  "6 Informational",
                  "7 Debugging"]
$log = event.get("sev")
$temp = $log.to_i
$i = 0
$y = 0
while ((($i+1)*8)<$temp) do 
  $i+=1 
end 
while (($i * 8) + ($y+1) != $temp) do 
  $y+=1
end
$message = category[$i] + " " + gravity[$y]
event.set("log description", $message)
'
    }
}
output {
      elasticsearch {
        	hosts => [ "localhost:9200" ]
   	     	user => elastic
       		password => elasticlourd
          index => "syslog-%{+YYYY.MM.dd}"
          }
      stdout { codec => rubydebug }
}

I have no idea what could be wrong because it worked until now ..

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.