Set data type to float (vs long) on filesystem.fs.used_p


(Josh Miller) #1

I am using logstash 2.2 and Elastic Search 1.5 on AWS and am not able to force the filesystem.fs.used_p value to be stored as a float so that I can check the filesystem used percentage properly.

I'm looking for help that would allow me to store this value as a float so that I can run ElastAlert against Elastic Search and alert on this field.

Amazon Linux
logstash-2.2.4-1.noarch
packetbeat-1.3.1-1.x86_64
topbeat-1.3.1-1.x86_64
filebeat-1.3.1-1.x86_64
Elastic Search: 1.5 - managed by AWS

  • I have also tried Elastic Search 5*

I've searched through the forums here and have not been able to find a solution.

I've tried:

  1. setting up a mutate section in the logstash config to force the conversion:

    filter {
    if [type] == "filesystem" {
    mutate {
    convert => { "filesystem.fs.used_p" => "float" }
    }
    mutate {
    convert => { "fs.used_p" => "float" }
    }
    mutate {
    convert => { "used_p" => "float" }
    }
    }
    }

As part of the mutate addition, I would stop logstash, delete the index, start logstash, and check to validate. It did not work, the field continued to be a long.

  1. I've used the mapping API to set the field to float:

    curl -XPUT "http://${ES_ENDPOINT}/${ES_INDEX}/_mapping/filesystem?ignore_conflicts=true" -d '
    {
    "filesystem" : {
    "properties" : {
    "fs" : {
    "properties" : {
    "used_p" : { "type" : "float", "store" : false }
    }
    }
    }
    }
    }
    '

That did not work either, or worked until logstash started logging to a new index for the following day.

curl -X GET https://${ES_ENDPOINT}/${ES_INDEX}/filesystem/_mapping?pretty
...
    "mappings" : {
      "filesystem" : {
...
          "fs" : {
            "properties" : {
...
              "used_p" : {
                "type" : "long"
              }
...

I would appreciate any help with this issue.

Thanks,
Josh


(Josh Miller) #2

I discovered my mistake on this issue. Nested fields must be addressed using brackets within mutate, ie:

        mutate {
                convert => [ "[fs][used_p]" , "float" ]
        }

Thanks,
Josh


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.