Logstash: mutate convert doesn't work on dynamic field name


(Cyril SANTUNE) #1

Hi,

I have a csv file as follow :
,10,CPU load,....
,3958,Memory used,....
,...,..,....

So, I parse the csv. I create a new field, I use content of the 3rd column
as name, and the 2nd column as value.

Extract of my configuration:

if [type] == "csv_perfmon" {
csv {
columns => [
"syslog_timestamp",
"value",
"name" ]
add_field => {
"syslog_severity" => "notice"
"syslog_severity_code" => "5"
"syslog_hostname" => "10.68.63.110"
"syslog_program" => "serverAgent"
"%{name}" => "%{value}"
}
}
date {
match => [ "syslog_timestamp", "UNIX_MS" ]
target => "syslog_timestamp"
}

mutate {
  rename => [ "message", "syslog_message" ]
  convert => [ "%{name}", "float" ]
  remove_field => [
    "value",

"name",
"column4",
"column5"
......... ]
}
}

Everything is fine in kibana/elasticsearch, I can see the new field and its
value.
However I can not change the type of the field.
The mutate part doesn't work :

mutate {
convert => [ "%{name}", "float" ]

The field type is still a string. So I can't graph it in kibana

Note: If I don't use a variable, it works
mutate {
convert => [ "CPU load", "float" ]

Is it a bug or I miss something ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/61a0c0d6-8748-4266-a810-5f51ce1829d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Gareth) #2

I'm seeing similar today with Logstash 5.2.
@Cyril_SANTUNE did you get an anwer to this?


(Gareth) #3

Answering my own question here.
I found this thread with the same issue. This is a known limitation and the answer is to use a bit of ruby to convert fields that have dynamic field names.

I'm going to try and find an example bit of ruby and post it back here.
And possibly here too.


(Cyril SANTUNE) #4

No, I don't have any answer. And I am not comfortable with Ruby so...


(system) #5