Ruby filter - how to access JSON fields when using the json codec

[anon24174512]

Dear all,

I have two payload.longitude and payload.latitude fields I can display in Kibana. There type in the elasticsearch mapping is float.

I want to divide them by 1000000 and set the values of the corresponding lat/lon of a geopoint.

Here is my logstash filter :

filter {
  ruby {
    code => "
      lat = event.get('[payload.latitude]').to_f / 10000000
      lon = event.get('[payload.longitude]').to_f / 10000000
      event.set('[coordinate][lat]', lat)
      event.set('[coordinate][lon]', lon)
        "
  }
}

This results in a geopoint with lat/lon equal to zero. I am using the json codec, so I believe the problem is how to access json fields ?

Thanks,

[paz]

Are those latitude/longitude fields supposed to be nested fields under payload in the original event? Unless it's a wrong field reference, or a typo in the field name it should work. I tested it on 5.6 as a single field (not nested) and works fine.

Btw, you can use .get methods inside .set to make your code a bit more concise

filter {
  ruby {
    code => "
      event.set('[coordinate][lat]', event.get('payload.latitude').to_f / 10000000)
      event.set('[coordinate][lon]', event.get('payload.longitude').to_f / 10000000)
    "
  }
}

[anon24174512]

Thanks for the answer, here an example of a json line :

{ "version" : "1", "payload" : { "longitude" : "193654098", "latitude" : "483751298" } }

So I guess payload is nested field or is the definition of nested different ?

[paz]

In that example the nested (children) fields are longitude and latitude, the payload is the parent field.

So in order to properly reference them, you need to do so like your coordinate field, as in [parent][child]

filter {
  ruby {
    code => "
      event.set('[coordinate][lat]', event.get('[payload][latitude]').to_f / 10000000)
      event.set('[coordinate][lon]', event.get('[payload][longitude]').to_f / 10000000)
    "
  }
}

[anon24174512]

Indeed I just tried it and it works. Thank you !

[anon24174512]

Just for any people looking for converting numbers to Geopoint, only two gigits after coma is allowed so it leads to the following :

filter {
ruby {
code => "
event.set('[coordinate][lat]', (event.get('[payload][latitude]').to_f / 10000000).round(2))
event.set('[coordinate][lon]', (event.get('[payload][longitude]').to_f / 10000000).round(2))
"
}
}