Logstash as SNMP Trap to GELF Converter, how to split message into seperate fields?


I am new to logstash and I want to use it to convert SNMP Traps to GELF.

This is my very first (simple) pipeline.

input {
  snmptrap {
    id => "snmptrap"
    type => "snmptrap"
    port => 1162
output {
  gelf {
    id => "gelf"
    host => "mygelfhost"
    port => 12201

This is working, but I get all Trap OIDs in one message.

#<SNMP::SNMPv2_Trap:xx @request_id=xx, @error_index=0, @error_status=0, @source_ip="x.x.x.x", @varbind_list=[
#<SNMP::VarBind:xx @name=[], @value=#<SNMP::TimeTicks:0x79d31b31 @value=7920439>>, 
#<SNMP::VarBind:xx @name=[], @value=[]>, 
#<SNMP::VarBind:xx @name=[], @value=#<SNMP::TimeTicks:0x28115c7e @value=1578645869>>, 
#<SNMP::VarBind:xx @name=[], @value="xxx">, 
#<SNMP::VarBind:xx @name=[], @value="xx">, 
#<SNMP::VarBind:xx @name=[], @value=#<SNMP::IpAddress:xx @value="xx">>, 
#<SNMP::VarBind:xx @name=[], @value=#<SNMP::Integer:0x15364a1e @value=9006>>, 
#<SNMP::VarBind:xx @name=[], @value=#<SNMP::Integer:0x35b30466 @value=3>>, 
#<SNMP::VarBind:xx @name=[], @value="xxxxx">]>

I want to have each splitted each OID into a seperate field.

If I use stdout as output, it looks very close. But with GELF (or syslog) output it does not. As I am very new to logstash I dont know how to proceed here. I think I have to use a filter to seperate this, but I dont know how to start. I played around with kv and split filters but I wasnt able to change this, now I am lost...

If anybody could me please point a direction to start?

Assuming that comes as a single event you could try starting with

# Extract the varbind_list
grok { match => { "message" => "@varbind_list=\[%{GREEDYDATA:[@metadata][varbind]}\]>$" } }
# and remove it
mutate { gsub => [ "message", "@varbind_list=\[.*\]>$", "@varbind_list=[]" ] }
# Trim the leading noise
mutate { gsub => [ "message", "^#<SNMP::SNMPv2_Trap:[^ ]+ ", "" ] }
kv { source => "message" field_split => "," value_split => "=" trim_key => " " }
ruby {
    code => '
        vb = event.get("[@metadata][varbind]")
        if vb
            # Extract the name and value pairs
            matches = vb.scan(/@name=([^ ]+), @value=([^,]+)(,|$)/)
            matches.each_index { |x|
                m2 = matches[x][1].scan(/@value=([^>]+)>/)
                # Is it "@value=#<SNMP::TimeTicks:0x28115c7e @value=1578645869>>" or "@value=7920439"
                if m2[0]
                    v = m2[0][0]
                    v = matches[x][1]
                # Remove trailing > and surrounding double quotes
                if v =~ />$/
                    v = v.gsub(/(.*)>$/, "\\1")
                if v =~ /^"(.*)"$/
                    v = v.gsub(/^"(.*)"$/, "\\1")
                event.set(matches[x][0], v)
            event.set("matches", matches)
1 Like

Hi Badger,

that looks very nice. I need some time to test and understand this. But at this time, thank you so much for taking your time and give me this configuration.

Edit: I did testing, its working great! I have to convert some values (IP address, timeticks), but I want to learn so I will figure it out. Again, thank you so much.