Set data type to float (vs long) on filesystem.fs.used_p

[itsecureadmin (Josh Miller)]

I am using logstash 2.2 and Elastic Search 1.5 on AWS and am not able to force the filesystem.fs.used_p value to be stored as a float so that I can check the filesystem used percentage properly.

I'm looking for help that would allow me to store this value as a float so that I can run ElastAlert against Elastic Search and alert on this field.

Amazon Linux
logstash-2.2.4-1.noarch
packetbeat-1.3.1-1.x86_64
topbeat-1.3.1-1.x86_64
filebeat-1.3.1-1.x86_64
Elastic Search: 1.5 - managed by AWS

• I have also tried Elastic Search 5*

I've searched through the forums here and have not been able to find a solution.

I've tried:

1. setting up a mutate section in the logstash config to force the conversion:

   filter {
   if [type] == "filesystem" {
   mutate {
   convert => { "filesystem.fs.used_p" => "float" }
   }
   mutate {
   convert => { "fs.used_p" => "float" }
   }
   mutate {
   convert => { "used_p" => "float" }
   }
   }
   }

As part of the mutate addition, I would stop logstash, delete the index, start logstash, and check to validate. It did not work, the field continued to be a long.

2. I've used the mapping API to set the field to float:

   curl -XPUT "http://${ES_ENDPOINT}/${ES_INDEX}/_mapping/filesystem?ignore_conflicts=true" -d '
   {
   "filesystem" : {
   "properties" : {
   "fs" : {
   "properties" : {
   "used_p" : { "type" : "float", "store" : false }
   }
   }
   }
   }
   }
   '

That did not work either, or worked until logstash started logging to a new index for the following day.

curl -X GET https://${ES_ENDPOINT}/${ES_INDEX}/filesystem/_mapping?pretty
...
    "mappings" : {
      "filesystem" : {
...
          "fs" : {
            "properties" : {
...
              "used_p" : {
                "type" : "long"
              }
...

I would appreciate any help with this issue.

Thanks,
Josh

[itsecureadmin (Josh Miller)]

I discovered my mistake on this issue. Nested fields must be addressed using brackets within mutate, ie:

        mutate {
                convert => [ "[fs][used_p]" , "float" ]
        }

Thanks,
Josh