Grok processor: Expected output to be decimal, but output was string

Elastic Stack Elasticsearch
1

I created a pipeline called parse_duration:

{
  "description" : "ParseDuration",
  "processors": [
    {
      "grok": {
        "field": "Duration",
        "patterns": ["%{NUMBER:DrillDuration}"]
      }
    }
  ]
}

I simulated posting some dummy data while invoking the pipeline:

{
  "docs": [
    {
      "_source": {
        "Duration": "4.13"
      }
    }
  ]
}

This was the response message:

{
    "docs": [
        {
            "doc": {
                "_index": "_index",
                "_type": "_doc",
                "_id": "_id",
                "_source": {
                    "DrillDuration": "4.13",
                    "Duration": "4.13"
                },
                "_ingest": {
                    "timestamp": "2019-05-08T08:13:35.339Z"
                }
            }
        }
    ]
}

When I posted some actual data and loaded them into Kibana, DrillDuration is considered to be a string value:

image

I want DrillDuration to be a decimal or a double. How should I fix my pipeline definition to achieve this?

2

See the grok basics paragraph in the docs. You need to specify the type in addition

The syntax for reusing a grok pattern comes in three forms: %{SYNTAX:SEMANTIC} , %{SYNTAX} , %{SYNTAX:SEMANTIC:TYPE} .

The TYPE is the type you wish to cast your named field. int , long , double , float and boolean are supported types for coercion.

You would need to use "%{NUMBER:DrillDuration:float}"

hope this helps

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.