Grok expression works in debugger but fails when posted on the pipeline simulation api

Elastic Stack Beats
1

Here is my simulate JSON

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "ASDF log pipeline",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            "%{TIMESTAMP_ISO8601:@timestamp} %{LOGLEVEL:level} %{WORD:namespace} .*? %{GREEDYDATA:message}"
          ]
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "@timestamp": "2018-12-13T20:58:12.651Z",
        "@metadata": {
          "beat": "filebeat",
          "type": "doc",
          "version": "6.2.1"
        },
        "message": "2018-12-12 10:15:29,697 DEBUG ASDF - OnLoadingDelayChangedCallback: 1, 2",
        "source": "C:\\ASDF\\serverlogs\\testlog.log",
        "offset": 158,
        "prospector": {
          "type": "log"
        },
        "beat": {
          "name": "ASDF",
          "hostname": "ASDF",
          "version": "6.2.1"
        }
      }
    }
  ]
}

This returns me the following error
"java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2018-12-12 10:15:29,697 DEBUG WayPoints - OnLoadingDelayChangedCallback: 1, 2]"

I am struggling to understand as the grok appear to be valid when tested in the debugger

2

You seem to have spaces before the colon in some of your pattern definitions. Can you try removing those?

3

I tried removing the spaces. The simulation still error.

4

Strange, because this work for me:

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "ASDF log pipeline",
    "processors": [
      {
        "grok": {
          "field": "message",
          "patterns": [
            "%{TIMESTAMP_ISO8601:@timestamp} %{LOGLEVEL:level} %{WORD:namespace} .*? %{GREEDYDATA:message}"
          ]
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "@timestamp": "2018-12-13T20:58:12.651Z",
        "@metadata": {
          "beat": "filebeat",
          "type": "doc",
          "version": "6.2.1"
        },
        "message": "2018-12-12 10:15:29,697 DEBUG ASDF - OnLoadingDelayChangedCallback: 1, 2",
        "source": "C:\\ASDF\\serverlogs\\testlog.log",
        "offset": 158,
        "prospector": {
          "type": "log"
        },
        "beat": {
          "name": "ASDF",
          "hostname": "ASDF",
          "version": "6.2.1"
        }
      }
    }
  ]
}
5

It does not work here. I just started a major update from v5.6.9 to v6.5.3 ill let you know if this help.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.