Unable to Grok scientific notation numbers using ingest pipelines

Elastic Stack Elasticsearch
1

Hi there,

I'm having the following issue with the Grok Processor in ingest pipeline as defined:

PUT _ingest/pipeline/number-pipe
{
  "processors" : [
    {
      "grok": {
        "field": "message",
        "patterns": [
			"number: %{NUMBER:test-number}"
        ]
      }
    }
  ]
}

Running the following _simulate:

POST _ingest/pipeline/number-pipe/_simulate
{
  "docs" : [
    {
      "_source": {
        "message": "number: 1.234e12"
      }
    }
  ]
}

Result:

{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_type": "_type",
        "_id": "_id",
        "_source": {
          "message": "number: 1.234e12",
          "test-number": "1.234"
        },
        "_ingest": {
          "timestamp": "2018-05-11T08:52:24.693Z"
        }
      }
    }
  ]
}

Am I doing the right thing using NUMBER as a grok pattern ?

Thanks

2

In case someone is having the same issue, I ended up using the following grok pattern:

PUT _ingest/pipeline/number-pipe
{
  "processors" : [
    {
      "grok": {
        "field": "message",
        "patterns": [
			"number: %{NUMBER_SCI:test-number}"
        ],
        "pattern_definitions" : {
          "NUMBER_SCI" : "%{NUMBER}(e%{NUMBER})?"
        }
      }
    },
    {
      "convert": {
        "field" : "test-number",
        "type": "float"
      }
    }
  ]
}

If anyone thinks there's a better way I'd be more than happy to know about it.
Thanks!

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.