Unable to use grok pattern on GREEDYDATA message

Hi,

I have this log message -
2022-03-08 04:16:04 [DEBUG] Creating linked clone: from Template-CentOS, to CentOS-001122

My logstash config. file is as follows -

filter {
    grok {
      match => {
        "message" => [
          "^%{TIMESTAMP_ISO8601:logTimestamp} \[%{LOGLEVEL:logLevel}\] %{DATA} %{DATA} %{DATA:actionName} %{GREEDYDATA:logMessage}$"
        ]
      }
    }

    if [actionName] == "clone:" {
      grok {
        match => {
          "logMessage" => [
            "^%{TIMESTAMP_ISO8601:logTimestamp} \[%{LOGLEVEL:logLevel}\] %{DATA} %{DATA} %{DATA:actionName} %{WORD} %{DATA:templateName} %{GREEDYDATA:logMessage}$"
          ]
        }
      }
    }
}

My actionName field contains string "clone:" and I have verified that it is entering the if loop using mutate filter (remove and rename fields are working). The only time I am having issue is when I am looking to grok for template name in the above copied log i.e. want to retrieve "Template-CentOS" name.

I have reviewed a lot of KB articles, however, couldn't get through. Can someone please point out what am I doing wrong?

Thanks,

Not quite sure your question but if all your logs follow a similar format then this should work.

^%{TIMESTAMP_ISO8601:logTimestamp} \[%{LOGLEVEL:logLevel}\] Creating linked %{DATA:actionName}: from %{DATA:from}\, to %{GREEDYDATA:to}

Output

{
  "logTimestamp": "2022-03-08 04:16:04",
  "logLevel": "DEBUG",
  "from": "Template-CentOS",
  "to": "CentOS-001122",
  "actionName": "clone"
}

Hi Aaron,

Thanks a lot for your response. Now that's where the challenge lies since there are several different events being logged in the files.

For example:

2022-03-28 07:05:31 [ERROR] save_variable: Error while trying to work with DB. Exception => (1205, u'Lock wait timeout exceeded; try restarting transaction')
2022-03-25 12:37:11: [DEBUG] diag-_do_action [1] action_name='alert' action_params='['admins', 'N5K failed to boot up']' ### dlgscr.py: 382: _do_action(): 
2022-03-08 04:16:04 [DEBUG] Creating linked clone: from Template-CentOS, to CentOS-001122

Regards,
Rahul

In addition to my previous comment, I am parsing different kind of events from logs. And I do segregating with different field names in Kibana visualisations.

For example:

2022-03-25 12:37:11: [DEBUG] diag-_do_action [1] action_name='alert' action_params='['admins', 'N5K failed to boot up']' ### dlgscr.py: 382: _do_action(): 

I am creating an alert visualisation in Kibana for the above event.

2022-03-08 04:16:04 [DEBUG] Creating linked clone: from Template-CentOS, to CentOS-001122

I am creating a template name visualisation for above event.

Note: The source of all these events is same.

Thanks,

If you want to parse all the different types of messages then you need to write grok patterns for all of them. It's not possible to write 1 to parse to the level you want.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.