Influxdb coerce_values is causing exception

Elastic Stack Logstash
1

I have logstash working fine with influxDB, but it is sending numeric values as strings so I can't use the influxDB aggregation functions like mean(). The solution to this seems to be coerce_values in the influxdb output, but when I try that, I get an exception from ruby, and I can't work out why.

Help would be much appreciated please.
Thanks,
Andrew

Example data:
"sensorID" => "59",
"indoorTemp" => "79.0",

Influxdb output defintion:
influxdb {
host => "127.0.0.1"
db => "test"
measurement => "sensor"
data_points => { "indoorTemp" => "%{indoorTemp}"
"sensorID" => "%{sensorID}"
}
send_as_tags => ["sensorID"]
coerce_values => { "indoorTemp" => "float"
"sensorID" => "integer"
}
}

Exception:
Failed to flush outgoing items {:outgoing_count=>1, :exception=>"NoMethodError", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:348:in escaped'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:226:inevents_to_request_body'", "org/jruby/RubyHash.java:1342:in each'", "org/jruby/RubyEnumerable.java:757:inmap'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:226:in events_to_request_body'", "org/jruby/RubyArray.java:2414:inmap'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:224:in events_to_request_body'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:171:inflush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:221:in buffer_flush'", "org/jruby/RubyHash.java:1342:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:216:in buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:159:inbuffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-influxdb-3.1.2/lib/logstash/outputs/influxdb.rb:165:in receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/outputs/base.rb:83:inmulti_receive'", "org/jruby/RubyArray.java:1613:in each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/outputs/base.rb:83:inmulti_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/output_delegator.rb:130:in worker_multi_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/output_delegator.rb:114:inmulti_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:301:in output_batch'", "org/jruby/RubyHash.java:1342:ineach'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:301:in output_batch'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:232:inworker_loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.1-java/lib/logstash/pipeline.rb:201:in `start_workers'"], :level=>:warn}

2

I think I have found the problem. There is a bug in the ruby code that only causes this problem if a field is defined as a tag - the test case in the 'spec' file doesn't set a tag so doesn't find this problem.

The tag value is being passed into the escaped() method, which tries to call gsub(), but the tag value has been changed from type string to type integer by the coerce_values, and an integer type clearly doesn't have a gsub() method. The fix is to call quoted(), rather than escaped().

IMHO the actual code fix is to change line 226 of V3.1.2 of influxdb.rb from:

result << "," << event["tags"].map { |tag,value| "#{escaped(tag)}=#{escaped(value)}" }.join(',') if event.has_key?("tags")

to

result << "," << event["tags"].map { |tag,value| "#{escaped(tag)}=#{quoted(value)}" }.join(',') if event.has_key?("tags")

Perhaps someone could verify and produce an official fix please (I edited the source file on my system and confirmed it worked).

Thanks,
Andrew

3

OK - please ignore this. I just realised that tag values are always stored as strings in InfluxDB, so should never be included in coerce_values. perhaps the plugin documentation could make that clear.
Andrew