Can convert "10%" string to float?

Elastic Stack Logstash
1

Hello,
It is possible to convert with filter logstash a string value like "10%" to float?

thnx.

2

Yes, you can do it with a ruby filter. This probably works:

event.set('fieldname', event.get('fieldname').gsub('%', '').to_f / 100)
3

my json data was like this { "payload" : { "humidity" : { "Id" : "Sensor_DHT", "Model" : "DHT11", "value" : "10%" } } }

can i use for 'fieldname' this '[payload][humidity][value]' ?

4

Why don't you try it out?

5

I have tried, but it doesn't work.
this is the code than i have tryed with your suggestion
ruby{ code => "event.set('[payload][humidity][value]', event.get('[payload][humidity][value]').gsub('%', '').to_f / 100)" }

I have tried even this code
mutate { gsub => [ "[payload][humidity][value]", "%", "." ] } mutate { convert => ["[payload][humidity][value]", "float" ] }
I have searched much in the web but don't know any other way to solve this.

6

"It doesn't work" isn't a useful error description. What happens? What does the resulting document look like? Copy/paste from Kibana's JSON tab.

7

for this code
ruby{ code => "event.set('[payload][humidity][value]', event.get('[payload][humidity][value]').gsub('%', '').to_f /100)" }
it shows me this error
[2018-03-26T16:08:13,745][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600} SyntaxError: (ruby filter code):1: syntax error, unexpected tREGEXP_BEG @codeblock = lambda { |event, &new_event_block| event.set('[payload][humidity][v alue]', event.get('[payload][humidity][value]').gsub('%', '').to_f /100) }^ eval at org/jruby/RubyKernel.java:1079 register at C:/Users/user/Desktop/Elastic/logstash-5.6.4/vendor/bundle /jruby/1.9/gems/logstash-filter-ruby-3.0.4/lib/logstash/filters/ruby.rb:38

8

but for this code
ruby{ code => " test = event.get('[payload][humidity][value]').gsub('%', '').to_f / 100 event.set('[payload][humidity][value]', test)" }

it shows me this error
[2018-03-26T16:13:40.921000 #7272] DEBUG -- : MONGODB | localhost:27017 | mon go_elastic.find | SUCCEEDED | 0.007s [2018-03-26T16:13:40,970][ERROR][logstash.filters.ruby ] Ruby exception occur red: undefined method 'gsub' for nil:NilClass

for this configuration it entered all my data but it does not switch the % character and doesn't convert it to float.

9

[logstash.filters.ruby ] Ruby exception occur red: undefined method 'gsub' for nil:NilClass

So event.get('[payload][humidity][value]') returned nil. This happens when the field doesn't exist.

10

Ok, i understand, so i think to use if statement to check if '[payload][humidity][value]' exist or not. Do you suggest another method?

11

now i used this code
if [payload][humidity][value]{ ruby{ code => " test = event.get('[payload][humidity][value]').gsub('%', '').to_f / 100 event.set('[payload][humidity][value]', test)" } }
it doesn't show any error and does not convert [payload][humidity][value] to float.

Capture

on kibana it was like this "payload_humidity_value": "10%",

12

I have solved the problem using this code:

if [payload_humidity_value] { mutate { gsub => [ "payload_humidity_value", "%", "" ] convert => ["payload_humidity_value", "float" ] } }

Seems like the problem was at the syntax. Calling the field in this way gsub => [ "payload_humidity_value", "%", "" ] solved it.

Notice that i am using Logstash_input_mongodb plugin to get data from MongoDB.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.