Elasticsearch grok processor error

Elastic Stack Elasticsearch
1

hi, can anybody tell me why this is error:

POST /_ingest/pipeline/_simulate
{
"pipeline": {
"description": "filebeat pipeline",
"processors": [
{
"grok": {
"field": "message",
"patterns": ["%{TIMESTAMP_ISO8601}:x"]
}
}
]
},
"docs": [
{
"_index": "index",
"_type": "_doc",
"_id": "id",
"_source": {
"message": "2017-10-12 13:12:32"
}
}
]
}

when i post this, the response is IllegalArgumentException, detailed error information is:

{
"docs": [
{
"error": {
"root_cause": [
{
"type": "exception",
"reason": "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 13:12:32]",
"header": {
"processor_type": "grok"
}
}
],
"type": "exception",
"reason": "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 13:12:32]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 13:12:32]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Provided Grok expressions do not match field value: [2017-10-12 13:12:32]"
}
},
"header": {
"processor_type": "grok"
}
}
}
]
}

thanks a lot

2

it's my fault, i should use %{TIMESTAMP_ISO8601:x} instead of %{TIMESTAMP_ISO8601}:x...

3

i found if i use elasticsearch grok processor, pattern TIMESTAMP_ISO8601 can not mach value "2017-10-12 1:12:32.232", but logstash grok can match. The follow is my use case

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "description": "filebeat pipeline",
  "processors": [
    {"grok": {
      "field": "message",
      "patterns": ["%{TIMESTAMP_ISO8601:logtime}"]
    }
    }]
  },
  "docs": [
    {
      "_index": "index",
      "_type": "_doc",
      "_id": "id",
      "_source": {
        "message": "2017-10-12 1:12:32.232"
      }
    }
    ]
}

and the resonse is:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "exception",
            "reason": "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 1:12:32.232]",
            "header": {
              "processor_type": "grok"
            }
          }
        ],
        "type": "exception",
        "reason": "java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 1:12:32.232]",
        "caused_by": {
          "type": "illegal_argument_exception",
          "reason": "java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [2017-10-12 1:12:32.232]",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Provided Grok expressions do not match field value: [2017-10-12 1:12:32.232]"
          }
        },
        "header": {
          "processor_type": "grok"
        }
      }
    }
  ]
}

but if i set message value as "2017-10-12 01:12:32.232", its ok. so the hour value must have two numbers in elasticsearch grok processor?

4

the ES grok processor is more strict here. You should try to avoid truncated representations of dates, as especially with different formatting this may lead to ambiguous dates.

5

ok，thanks