I have an ingest pipeline that I want to use to extract specific errors and the date from and Oracle alert log. I have it working if the message contains an error, but not when it doesn't. Please see below:
post _ingest/pipeline/_simulate
{
"pipeline":{
"description":"grok test",
"processors":[
{
"grok":{
"trace_match":true,
"ignore_missing":true,
"field":"message",
"patterns":[
"%{ORADATE:ora-date}(.|\n)*%{ORAERR:ora-err}",
"%{ORADATE:ora-date}"
],
"pattern_definitions":{
"ORADATE":"[A-z]{3} [A-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [0-9]{4}",
"ORAERR":"ORA-[0-9]{5}"
}
}
}
]
},
"docs":[
{
"_source":{
"message":"Wed Nov 30 14:34:48 2016\nORA-01555 caused by SQL statement below (SQL ID: xxxxxxxxxxxxxxxxx, Query Duration=0 sec, SCN: 0x0030.093d9a86):\nSELECT * FROM TABLE"
}
}
]
}
This creates two new fields, ora-date and ora-err filled out correctly. If I edit the part of the doc ORA-01555 so that it doesn't match the ORAERR regex, it doesn't even match the date. See response below:
{
"docs": [
{
"doc": {
"_type": "_type",
"_id": "_id",
"_index": "_index",
"_source": {
"message": "Wed Nov 30 14:34:48 2016\nORA-a01555 caused by SQL statement below (SQL ID: xxxxxxxxxxxxxxxxx, Query Duration=0 sec, SCN: 0x0030.093d9a86):\nSELECT * FROM TABLE"
},
"_ingest": {
"_grok_match_index": "1",
"timestamp": "2016-12-07T14:00:28.783+0000"
}
}
}
]
}
One further thing to point out: "_grok_match_index": "1" indicates that it is matching the second value of the patterns array, but ora-date is not being set? What am I missing here?