Logstash Ruby filter error

Elastic Stack Logstash
1

Hello all,

Running Logstash v7.8.1. Ruby codec v3.1.5.

Been playing around using the ruby filter plugin. Great filter plugin btw!!

I have a couple of logstash confs that have small ruby script filters. While all the filters seem to be working. I still see the below error log when looking at the logstash pipeline logs.

[ERROR][logstash.filters.ruby    ][pipeid] Ruby exception occurred: undefined method `+' for nil:NilClass

When looking into this, the below are the only scripts that I can see include the parameters that would cause this.

Script 1

ruby {
  code => "
    e = event.get('[geo_src][ip]')
      url = 'https://www.virustotal.com/gui/search/'
      l = url + e
    event.set('[geo_src][ip_reputation]', l)
  "
}

Script 2

ruby {
  code => "
    i = event.get('[dst_ip]')
      a = i.to_s
      b = a.split('.')
      c = b[0] + '.' + b[1] + '.' + b[2]
     event.set('[dst_ip_grouping]', c)
  "
}

Data is still being seen in Kibana and all, but would this mean that there would be data from this that has not been/ unable to have been filtered as intended and dropped etc?

Would there be a work around for this or a way to modify the scripts as I am unsure why this is happen? When checking the scripts in the online ruby tool (Repl.it)

Any help would be greatly appreciated.
Thank you!

2

I think the error comes from this part because a didn't contain any dots, so the split result only has one entry. Do you maybe have some IPV6 values in dst_ip?

3

Hey @Jenni

Thank you very much for replying to my message and your help!

Yes, I was thinking this but I have tried a couple of different filters that happen before and are included within if statements etc but am still getting this.

Examples of this are below:

ex 1

if [dst_ip] === %{IPV6} {
    mutate {
        add_tag => "IPv6"
        ...
    }
} else {
ruby {
      code => "
        i = event.get('[dst_ip]')
          a = i.to_s
          b = a.split('.')
          c = b[0] + '.' + b[1] + '.' + b[2]
         event.set('[dst_ip_grouping]', c)
      "
    }
}

ex 2

if [dst_ip] === %{IPV6} {
    mutate {
        add_tag => "IPv6"
        tag_on_failure => ["non_ipv6"]
        ...
    }
} 
if "non_ipv6" in [tags] {
ruby {
      code => "
        i = event.get('[dst_ip]')
          a = i.to_s
          b = a.split('.')
          c = b[0] + '.' + b[1] + '.' + b[2]
         event.set('[dst_ip_grouping]', c)
      "
    }
}

ex 3

ruby {
      code => "
        i = event.get('[dst_ip]')
          a = i.to_s
          b = a.split('.')
          c = b[0] + '.' + b[1] + '.' + b[2]
         event.set('[dst_ip_grouping]', c)
      "
      tag_on_expection => "ipv6_error_ruby"
    }
if "ipv6_error" in [tags] {
    mutate {
        remove_tag => "ipv6_error_ruby"
        ...
    }
}

Thinking a better way of this if [dst_ip] === %{IPV6} { would be cleaner?

Maybe I've been looking at this for a while and missing what is there? Maybe the logic is incorrect?
Thinking that if I put a filter beforehand to separate IPv4 and IPv6 addresses coming through.Then only the IPv4 addresses with run through the ruby script (bar ex 3).

Just want to be able to clean up the pipeline logs.

4

And Logstash didn't crash with this configuration? I don't know where you got that, but that kind of syntax and functionality doesn't exist. You could follow this example: How to check client_ip ipv4 or ipv6? - #3 by Badger (Additionally you should maybe check that the field isn't an empty string.)

And it's tag_on_exception, not tag_on_expection. But that only changes the tag, it doesnt suppress the error message.

(And just one more comment: You can shorten your IP with just one line of code: in ruby how to remove the last part of ip address - Stack Overflow)

5

Whoops! A couple of typos in my last message. I do apologize for that!!! :see_no_evil:

Ahhhh, yes have now added the script and tested and is now not showing the error messages in the log as before.

Updated script

if ":" in [dst_ip] {
    mutate {
        add_tag => "IPv6"
    }
} else if [dst_ip] == "" {
        mutate {
          add_tag => "no_ip_found"
        }
} else {
ruby {
      code => "
       i = event.get('[dst_ip]')
         b = i.to_s.split('.')[0...-1].join('.')
       event.set('[dst_ip_grouping]', b)
      "
    }
}

Thank you very much for your help, speedy replies and the push in the correct direction :partying_face: :smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.