Ingest pipepine not extracting date when matched on second pattern


#1

I have an ingest pipeline that I want to use to extract specific errors and the date from and Oracle alert log. I have it working if the message contains an error, but not when it doesn't. Please see below:

post _ingest/pipeline/_simulate
{  
   "pipeline":{  
      "description":"grok test",
      "processors":[  
         {  
            "grok":{  
               "trace_match":true,
               "ignore_missing":true,
               "field":"message",
               "patterns":[  
                  "%{ORADATE:ora-date}(.|\n)*%{ORAERR:ora-err}",
                  "%{ORADATE:ora-date}"
               ],
               "pattern_definitions":{  
                  "ORADATE":"[A-z]{3} [A-z]{3} [0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2} [0-9]{4}",
                  "ORAERR":"ORA-[0-9]{5}"
               }
            }
         }
      ]
   },
   "docs":[  
      {  
         "_source":{  
            "message":"Wed Nov 30 14:34:48 2016\nORA-01555 caused by SQL statement below (SQL ID: xxxxxxxxxxxxxxxxx, Query Duration=0 sec, SCN: 0x0030.093d9a86):\nSELECT * FROM TABLE"
         }
      }
   ]
}

This creates two new fields, ora-date and ora-err filled out correctly. If I edit the part of the doc ORA-01555 so that it doesn't match the ORAERR regex, it doesn't even match the date. See response below:

{
  "docs": [
    {
      "doc": {
        "_type": "_type",
        "_id": "_id",
        "_index": "_index",
        "_source": {
          "message": "Wed Nov 30 14:34:48 2016\nORA-a01555 caused by SQL statement below (SQL ID: xxxxxxxxxxxxxxxxx, Query Duration=0 sec, SCN: 0x0030.093d9a86):\nSELECT * FROM TABLE"
        },
        "_ingest": {
          "_grok_match_index": "1",
          "timestamp": "2016-12-07T14:00:28.783+0000"
        }
      }
    }
  ]
}

One further thing to point out: "_grok_match_index": "1" indicates that it is matching the second value of the patterns array, but ora-date is not being set? What am I missing here?


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.