Mutate filter throws NumberFormat Exception for no reason

I recently migrated from 3.1 to 5.1 and wanted to continue to use my logstash filter/output configuration. Since the event API changed I adopted those but soon realised that the mutate filter is suddenly throwing excpetions that haven't been there before!

In 3.1 I had a filter that looked like this:

if ( [businessTransactions][type] == 1 ) {
  mutate {
    add_field => {
      "clientTime" => '%{[businessTransactions][occurrences][clientTime]}'
      "serverTime" => '%{[businessTransactions][occurrences][serverTime]}'
    }
  }
}

This gave me float values in the fields that were nicely written to influxDB afterwards.
First I realised that before there was nice automatic conversion to float and with 5.1 these float values were written as string now. So i added a mutate convert filter

if ( [businessTransactions][type] == 1 ) {
  mutate {
    add_field => {
      "clientTime" => '%{[businessTransactions][occurrences][clientTime]}'
      "serverTime" => '%{[businessTransactions][occurrences][serverTime]}'
    }
  }
  mutate {
    convert => {
      "serverTime" => "float"
      "clientTime" => "float"
    }
  }
}

Doing so 'sometimes' gives me a Exception in syslog. I can guarantee that the values in clientTime are always numbers and no strings. This is the top stacktrace:

Exception in thread "[main]>worker3" java.lang.NumberFormatException: For input string: "clientTime"
#011at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
#011at java.lang.Integer.parseInt(Integer.java:580)
#011at java.lang.Integer.parseInt(Integer.java:615)
#011at org.logstash.Accessors.fetch(Accessors.java:130)
#011at org.logstash.Accessors.get(Accessors.java:20)
#011at org.logstash.Event.getUnconvertedField(Event.java:160)
#011at org.logstash.Event.getField(Event.java:150)
#011at org.logstash.KeyNode.evaluate(KeyNode.java:26)
#011at org.logstash.StringInterpolation.evaluate(StringInterpolation.java:47)
#011at org.logstash.Event.sprintf(Event.java:294)
#011at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_sprintf(JrubyEventExtLibrary.java:200)
...
#011at rubyjit.LogStash::Filters::Mutate$$filter_7ece4ecf2c9d8324f6c4933fda087138f0104c701442407170.__file__(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/logstash/filters/mutate.rb:216)
#011at rubyjit.LogStash::Filters::Mutate$$filter_7ece4ecf2c9d8324f6c4933fda087138f0104c701442407170.__file__(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-mutate-3.1.3/lib/logstash/filters/mutate.rb)
#011at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:201)
#011at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:177)
#011at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:183)
#011at rubyjit.LogStash::Filters::Base$$do_filter_8e8403dcfdf01a35ffca12ed35ec4e79455489071442407170.__file__(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145)
#011at rubyjit.LogStash::Filters::Base$$do_filter_8e8403dcfdf01a35ffca12ed35ec4e79455489071442407170.__file__(/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb)
#011at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:201)

So I added the conversion and field creation directly in a ruby filter:

if ( [businessTransactions][type] == 1 ) {
  ruby {
    code => "
      unless event.get('[businessTransactions][occurrences][clientTime]').nil?
      event.set('clientTime',sprintf('%.4f',event.get('[businessTransactions][occurrences][clientTime]')).to_f)
    end
      unless event.get('[businessTransactions][occurrences][serverTime]').nil?
      event.set('serverTime',sprintf('%.4f',event.get('[businessTransactions][occurrences][serverTime]')).to_f)
    end
    "
  }
}

This seems to work but I have other filters where I now get NumberFormat Exceptions that do not make sense at all. e.g. this mutate:

mutate {
  add_field => {
    "tagging_values" => "%{[businessTransactions][occurrences][dimensions]}"
    "application" => "%{[businessTransactions][application]}"
    "businesstransaction" => "%{[businessTransactions][name]}"
    "systemprofile" => "%{[businessTransactions][systemProfile]}"
    "project" => "%{[businessTransactions][systemProfile]}"
    "responsetime" => '%{[businessTransactions][occurrences][responseTime]}'
    "time" => "%{[businessTransactions][occurrences][startTime]}"
    "resultmeasure_names" => "%{[businessTransactions][measureNames]}"
    "resultmeasure_values" => "%{[businessTransactions][occurrences][values]}"
  }
}

It now also throws an exception after I've fixed the above.:

Exception in thread "[main]>worker1" java.lang.NumberFormatException: For input string: "dimensions"
#011at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
#011at java.lang.Integer.parseInt(Integer.java:580)
#011at java.lang.Integer.parseInt(Integer.java:615)
#011at org.logstash.Accessors.fetch(Accessors.java:130)
#011at org.logstash.Accessors.get(Accessors.java:20)
#011at org.logstash.Event.getUnconvertedField(Event.java:160)
#011at org.logstash.Event.getField(Event.java:150)
#011at org.logstash.KeyNode.evaluate(KeyNode.java:26)

The data I'm feeding into logstash is sane and correct, as soon as I hook up my event source to logstash 3.1 everything works fine.

Any ideas? I've seen that there are other posts about mutate filter problems...

Thanks,
Reinhard

Please show an example event, preferably using a stdout { codec => rubydebug } output.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.