# Logstash to convert scientific notation to float

**URL:** https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231
**Category:** Logstash
**Created:** [October 10, 2022, 1:46pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231 "2022-10-10T13:46:58Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![the\_elkguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the_elkguy/32/111890_2.png) [@the\_elkguy](https://discuss.elastic.co/u/the_elkguy)
#### Post date: [October 10, 2022, 1:46pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/1 "2022-10-10T13:46:58Z")

</div>

Hello,

I'm using logstash ruby filter to convert scientific notation from string to float which seem to be not working. I have used GROK to extract the scientific notation (1.989e-04) from the message and now converting to real number (0.0001989).  
**The values i get is just 0**

Here is my code.

```auto
        ruby {
            code => "
                require 'bigdecimal'
                event.set('sourceOffset', ('%.20f' % (BigDecimal.new event.get('sourceOffset'))).sub(/\.?0*$/, '').to_f)
            "

```

Also the **logstash error:**

[2022-10-10T09:09:08,123][ERROR][logstash.filters.ruby] Ruby exception occurred: invalid value for BigDecimal(): "-" {:class=\>"ArgumentError", :backtrace=\>["org/jruby/ext/bigdecimal/RubyBigDecimal.java:716:in `new'", "(ruby filter code):4:in `block in filter\_method'", "/apps/elk/logstash-7.17.1/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:96:in `inline_script'", "logstash-7.17.1/vendor/bundle/jruby/2.5.0/gems/logstash-filter-ruby-3.1.8/lib/logstash/filters/ruby.rb:89:in `filter'", "logstash-7.17.1/logstash-core/lib/logstash/filters/base.rb:159:in `do_filter'", "logstash-7.17.1/logstash-core/lib/logstash/filters/base.rb:178:in `block in multi\_filter'", "org/jruby/RubyArray.java:1821:in `each'", "logstash-7.17.1/logstash-core/lib/logstash/filters/base.rb:175:in `multi\_filter'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134:in `multi_filter'", "/logstash-7.17.1/logstash-core/lib/logstash/java_pipeline.rb:299:in `block in start\_workers'"]}

Appreciate some help.  
thanks,

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 10, 2022, 4:34pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/2 "2022-10-10T16:34:48Z")

</div>

If your [sourceOffset] field contains "1.989e-04" then your ruby filter will produce

```
"sourceOffset" => 0.0001989,

```

However, you have no exception handling. The `Ruby exception occurred: invalid value for BigDecimal(): "-"` occurs if the [sourceOffset] field contains `-`.

You could use try/catch or test whether [sourceOffset] is in scientific notation using a [regexp](https://stackoverflow.com/questions/50528957/regex-to-match-scientific-notation-of-numbers-but-not-other-letters).

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [October 10, 2022, 4:45pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/3 "2022-10-10T16:45:24Z")

</div>

Have you tried with mutate convert?

```auto
mutate {
        convert => {
          "sourceOffset" => "float"
        }
}

```

---

<div class="post-metadata">

### Author: ![the\_elkguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the_elkguy/32/111890_2.png) [@the\_elkguy](https://discuss.elastic.co/u/the_elkguy)
#### Post date: [October 17, 2022, 2:58pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/4 "2022-10-17T14:58:52Z")

</div>

@Badger

Yes that's working. I had to change to GROK patterns as %{USERNAME:sourceoffset}  
then i ran the Ruby without the (.to\_f) which worked.  
the values are now coming as String.  
Ruby failed to convert that to Float and so many error;s about the exceptions.

@Rios

I also treid to convert that to float, that actually shows back the scientific notation value again. so i removed that.

The only issue now is how do I convert it to Number type.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [October 17, 2022, 3:04pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/5 "2022-10-17T15:04:07Z")

</div>

Number as integer or number as type?  
If is as the type, it's already, and you have recreate mappings in Kibana.

---

<div class="post-metadata">

### Author: ![the\_elkguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the_elkguy/32/111890_2.png) [@the\_elkguy](https://discuss.elastic.co/u/the_elkguy)
#### Post date: [October 17, 2022, 3:54pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/6 "2022-10-17T15:54:17Z")

</div>

@Rios

Apparently its not  
Ruby is converting as String.

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [October 17, 2022, 4:01pm UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/7 "2022-10-17T16:01:36Z")

</div>

When is converted to integer or float it's number. Use debug, will see values.

---

<div class="post-metadata">

### Author: ![the\_elkguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/the_elkguy/32/111890_2.png) [@the\_elkguy](https://discuss.elastic.co/u/the_elkguy)
#### Post date: [October 19, 2022, 9:25am UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/8 "2022-10-19T09:25:54Z")

</div>

My ruby is converting the scientific notation in string but i need in float  
i need help here,  
Mutate filter convert is showing back the scientific notation in number type # ( -1.038e-06 ) but not in decimal -0.000001038.

```auto
 ruby {
                        code => "
                        require 'bigdecimal'
                        event.set('chronyoffset', ('%.20f' % (BigDecimal.new event.get('chronyoffset'))).sub(/\.?0*$/, ''))
                        "
                        }

```

Please provide me correct ruby code to fix this.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 16, 2022, 9:26am UTC](https://discuss.elastic.co/t/logstash-to-convert-scientific-notation-to-float/316231/9 "2022-11-16T09:26:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
