Create a calculated field or empty if value missing

Hi here,

I'm struggling trying to find the best way to create a field been calculated by doing the multiplication of two others. So first, I need to test that both fields are not null and then I can do my multiplication. The important thing is at this end, I need to have this field created anyway, populated with a value or null. So far what I have :

if ![coef] or ![int1] { mutate { add_field => { "int1coef" => "" } } } else { ruby { code => "event.set('int1coef', event.get('int1') * event.get('coef'))" } }

Which worked fine when int1 and coef not null, but doesn't when int1 is null, int1coef is an empty string in my kibana index and not tagged as null with the "-" ...
Does someone has a better idea ?

Thanks
Regards
Guillaume

Your filter is explicitly setting it to an empty string when either value is false-y (e.g., not present).

If I needed a "null" value to be set, I would do something like:

mutate {
  ruby {
    code => "
      int1 = Integer(event.get('int1'))
      coef = Integer(event.get('coef'))
      value = (int1 * coef) unless int1.nil? || coef.nil?
      event.set('int1coef', value)
    "
  }
}

If I didn't need a value to be set if one of the inputs were missing, I would do:

if [int1] and [coef]
mutate {
  ruby {
    code => "
      int1 = Integer(event.get('int1'))
      coef = Integer(event.get('coef'))
      value = (int1 * coef)
      event.set('int1coef', value)
    "
  }
}

If the source fields could hold something that had decimal precision, I would use Float() instead of Integer() to extract the relevant bits.

Hi @yaauie and thanks for your very helpful hand

I tried your first solution as I want my field created at the end anyway - null or populated with a float
After all, I've got this issue :

Ruby exception occurred: can't convert nil into Float
As said before intx can be null (empty value from database) or integer/float

My conf looks like that :

filter {
    ruby { code => "
           coef = Float(event.get('coef'))
           int1 = Float(event.get('int1'))
           int2 = Float(event.get('int2'))
           int3 = Float(event.get('int3'))
           int4 = Float(event.get('int4'))
           int5 = Float(event.get('int5'))

           myint1 = (int1* coef) unless int1.nil? || coef.nil?
           myint2 = (int2* coef) unless int2.nil? || coef.nil?
           myint3 = (int3* coef) unless int3.nil? || coef.nil?
           myint4 = (int4* coef) unless int4.nil? || coef.nil?
           myint5 = (int5* coef) unless int5.nil? || coef.nil?

           event.set('int1coef', myint1)
           event.set('int2coef', myint2)
           event.set('int3coef', myint3)
           event.set('int4coef', myint4)
           event.set('int5coef', myint5)
        " }
}   

I also had to remove your "mutate" at the begging, seemed to be wrong
Thanks again
Guillaume

I'll go for something like below :

event.get('coef') == nil || event.get('int1') == nil ? event.set('int1coef', nil) : event.set('int1coef', event.get('coef') * event.get('int1'))

What do you think about it ?

You're right; my suggestion was fragile against empty inputs. This should be better, because it only passes the value to Kernel#Float() if it is non-null.

filter {
    ruby { code => "
           coef = event.get('coef')
           int1 = event.get('int1')
           int2 = event.get('int2')
           int3 = event.get('int3')
           int4 = event.get('int4')
           int5 = event.get('int5')

           coef = Float(coef) unless coef.nil?
           myint1 = (Float(int1)* coef) unless int1.nil? || coef.nil?
           myint2 = (Float(int2)* coef) unless int2.nil? || coef.nil?
           myint3 = (Float(int3)* coef) unless int3.nil? || coef.nil?
           myint4 = (Float(int4)* coef) unless int4.nil? || coef.nil?
           myint5 = (Float(int5)* coef) unless int5.nil? || coef.nil?

           event.set('int1coef', myint1)
           event.set('int2coef', myint2)
           event.set('int3coef', myint3)
           event.set('int4coef', myint4)
           event.set('int5coef', myint5)
        " }
} 

Hi @yaauie, thanks for coming back again

Your solution works fine, thanks. Have you seen mine above too ?

Was wondering if it can work as well ?

Guillaume

In Ruby, testing an object to see if it is nil is typically done with Object#nil?, instead of with Object#==(other); your code will likely work, but a more idiomatic version of it would be:

(event.get('coef').nil? || event.get('int1').nil?) ? event.set('int1coef', nil) : event.set('int1coef', event.get('coef') * event.get('int1'))

This could be DRY'd up a little bit to be:

event.set('int1coef', (event.get('coef').nil? || event.get('int1').nil?) ? nil :  event.get('coef') * event.get('int1'))

Oh the last one looks/works perfect ! Thanks a lot
Have a great day
Can close it

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.