Convert $0.50 to float converts it to 0.00

[Yaniv_Nuriel]

Hey all,

Here are my load file:

Spend,Bbb
$0.55,100
$0.80,200

The conf.file is:

input {
        file {
            path => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-15.txt"
            start_position => "beginning"
            sincedb_path => "NUL"
        }
}
filter{

    csv {
        separator => ","
        autogenerate_column_names => true

    	columns => ["Spend", "Bbb"]
    	 
    }
    mutate{
        convert => ["Spend","float"]
    }
}

output {
    stdout { }
}

and the output is:

Yanivs-MBP:logstash yanivnuriel$ logstash -f test.conf 

Sending Logstash logs to /usr/local/Cellar/logstash-full/7.6.2/libexec/logs which is now configured via log4j2.properties
[2020-04-17T21:57:33,742][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-04-17T21:57:34,760][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.6.2"}
[2020-04-17T21:57:45,914][INFO ][org.reflections.Reflections] Reflections took 209 ms to scan 1 urls, producing 20 keys and 40 values 
[2020-04-17T21:57:57,234][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been created for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-04-17T21:57:57,376][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["/Users/yanivnuriel/logstash/test.conf"], :thread=>"#<Thread:0x51fc8441 run>"}
[2020-04-17T21:58:03,907][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-04-17T21:58:04,134][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-04-17T21:58:04,153][INFO ][filewatch.observingtail  ][main] START, creating Discoverer, Watch with file and sincedb collections
[2020-04-17T21:58:06,089][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
/usr/local/Cellar/logstash-full/7.6.2/libexec/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
          "host" => "Yanivs-MBP",
           "Bbb" => "100",
       "message" => "$0.55,100\r",
      "@version" => "1",
    "@timestamp" => 2020-04-17T18:58:07.550Z,
          "path" => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-15.txt",
         "Spend" => 0.0
}
{
          "host" => "Yanivs-MBP",
           "Bbb" => "Bbb",
       "message" => "Spend,Bbb\r",
      "@version" => "1",
    "@timestamp" => 2020-04-17T18:58:07.431Z,
          "path" => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-15.txt",
         "Spend" => 0.0
}
{
          "host" => "Yanivs-MBP",
           "Bbb" => "200",
       "message" => "$0.80,200\r",
      "@version" => "1",
    "@timestamp" => 2020-04-17T18:58:07.554Z,
          "path" => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-15.txt",
         "Spend" => 0.0

As you can see the value in Spend is 0.0 instead of 0.55 and 0.8.

Thanks
Yaniv

[Badger]

Use mutate+gsub to remove the $ sign before the convert.

mutate { gsub => [ "Spend", "\$", "" ] }

[Yaniv_Nuriel]

Thanks Badger, but I get the same result for this line:

    mutate{
        gsub => [ "Spend", "\$", "" ]
        convert => ["Spend","float"]
    }

B.T.W. when I remove the $ from the load file it works fine, so indeed the $ causes the mass

[Badger]

A mutate filter performs operations in a fixed order. convert executes before gsub, so when the gsub executes [Spend] is already equal to 0.0. Split that into two mutate filters.

[Yaniv_Nuriel]

Understood!
You're the best

This is the fix

    mutate{
        gsub => [ "Spend", "\$", "" ]
    }
    mutate{
        convert => ["Spend","float"]
    }

Here is the output:

{
      "@version" => "1",
    "@timestamp" => 2020-04-18T07:13:14.133Z,
       "message" => "$0.80,200\r",
          "host" => "Yanivs-MBP",
          "path" => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-17.txt",
           "Bbb" => "200",
         "Spend" => 0.8
}
{
      "@version" => "1",
    "@timestamp" => 2020-04-18T07:13:14.131Z,
       "message" => "$0.55,100\r",
          "host" => "Yanivs-MBP",
          "path" => "/Users/yanivnuriel/logstash/data/ads/searchterms/us/one-line-17.txt",
           "Bbb" => "100",
         "Spend" => 0.55
}