After add_field, conversion from string to float unsuccessful

Elastic Stack Logstash
1 
filter {
grok { match => { "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:min}:%{SECOND:sec} \[%{DATA:pool}\] %{DATA:metric} %{DATA:metricname} %{DATA:datatype} %{NUMBER:metricvalue:float} %{GREEDYDATA:unit}" } }

  mutate {
          copy => { "source" => "source_tmp" }
         }
 mutate {
    add_field => { "metric_%{metricname}" => "%{metricvalue}" }
        }
 mutate {
    convert => { "metric_%{metricname}" => "float" }
        }
  mutate {
          split => ["source_tmp", "/"]
          add_field => { "applicationID" => "%{[source_tmp][4]}" }
         }
}
2

Correct. The convert function does not sprintf the LHS, so you cannot use field references. There are two open issues for that, here and here.

3

@Badger Thanks for your prompt reply.
What should be the work around for it?..

4

@magnusbaeck any work arounds for this?

5

The workaround would be to do it in ruby. Possibly with a regexp to match the key.

6

@Badger any example to refer?

7 
    ruby {
        code => '
            event.to_hash.each { |k, v|
                if k =~ /^metric_/
                    if v.to_f.to_s == v
                        event.set(k, v.to_f)
                    end
                end
            }
        '
    }
8

@Badger Thanks a lot i will try this filter in my configuration.
This requires KV filter? Because we have split the fields using grok.
message format is -------
"2019-07-23 08:00:15.965 [pool-2-thread-1] metric Address_DATABASE_SEARCH_Country_in int32 7 count"

10

@Badger
@magnusbaeck
if i use this
input {
beats {
port => 5044
}
}

filter {
grok { match => { "message" => "%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} %{HOUR:hour}:%{MINUTE:min}:%{SECOND:sec} \[%{DATA:pool}\] %{DATA:metric} %{DATA:metricname} %{DATA:datatype} %{NUMBER:metricvalue:float} %{GREEDYDATA:unit}" } }

  mutate {
          copy => { "source" => "source_tmp" }
         }
  mutate {
          add_field => { "message_tmp" => "metric_%{metricname} = %{metricvalue} %{unit}"}
  }
 grok { match => { "message_tmp" => "metric_%{DATA:metricname} = %{NUMBER:value:float}" } }
 # mutate {
  #  add_field => { "metric_%{metricname}" => "%{metricvalue}" }
  #      }
 #mutate {
    #convert => { "metric_%{metricname}" => "float" }
    #    }
  mutate {
          split => ["source_tmp", "/"]
          add_field => { "applicationID" => "%{[source_tmp][4]}" }
         }  
}
output {
  elasticsearch {
    hosts => ["xyz:9200"]
    index => "%{[applicationID]}-%{+YYYY.MM.dd}"
  }
}
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.