Grok: how to handle not found patterns


Hello Members,

I'm receiving this string as input event:
2019-02-13 19:20:07,659 INFO 00ACDD5C000BAFFDC4D1353F49568680 [main] TEST (some_text)

and I'm using pretty simple pattern:

grok {
# grab the date, log level, gateway name and the component GUID.
match => { "message" => "%{TIMESTAMP_ISO8601:date} \s*.* %{WORD:GUID} [.*] %{WORD:GW}" }
# construct a file name.
add_field => { "filename" => "%{[GW]}_%{[GUID]}" }

In a nutshell, I'm capturing:

  1. Timestamp
  2. Ignoring the log level
  3. GUID
  4. Ignoring anything inside a square brace
  5. A word as GW

The issue is, in the above log message, I may or may not get the word TEST. In short I might receive an event similar to:

2019-02-13 19:20:07,659 INFO 00ACDD5C000BAFFDC4D1353F49568680 [main]  (some_text)

If this happens, grok fails to parse anything.

As you could see, the TEST is missing from the log. In this case, I would like to either:

  1. populate GW as an empty string, ""
  2. have (somehow) this:

add_field => { "filename" => "%{[GUID]}" }

Is this possible?


You can make the GW field optional using

match => { "message" => "%{TIMESTAMP_ISO8601:date} \s*.* %{WORD:GUID} \[.*\] (%{WORD:GW})?" }

Then you can check to see if the field exists and take appropriate action

if [GW] { ...