How to specify a default value for my field in grok pattern match

[ameeto17 (Ameet Castell)]

I have a pattern to match using GROK
Dec 14 03:13:01 ppddc1kfep302 my-checker: Context SHA of VSP Logger Software da39a3ee5e6b4b0d3255bfef95601890afd80709

I have the format below for the match
%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:my_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software} %{DATA:rel_version} %{WORD:sha}

This works but as you can see rel_version would be blank above but i want to specify that as a defualt value of 1.0.1. How can i do that?

Thanks in anticipation

[shrikantgulia (Shrikant)]

Hi, you can provide through mutate filter plugin

Mutate filter plugin | Logstash Reference [8.11] | Elastic

hope it works!

[Rios]

Partly true Shrikant. He didn't mention what he is trying by specifying several lines to figure out the pattern for:

Dec 14 03:13:01 ppddc1kfep302 my-checker: Context SHA of VSP Logger Software da39a3ee5e6b4b0d3255bfef95601890afd80709
Dec 12 12:01:27 ppdtest302 test-checker: Context SHA of TEST Software Version 3.0.1_RC5 0b1f71223180bf0df9330b13e17f8d7c62dfdaad16b97a80b8a25c99409c1109

This pattern is working:

filter {

   grok {
     match => { "message" => "%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:my_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software} Software (%{DATA})?%{SPACE}(%{DATA:version})?%{SPACE}%{WORD:hash}" }
   }
   if (![version]){
   mutate {  add_field => {"version" => "1.0.1"} }
   }
}

I assumed that version is an option filed as well as "Version" text. You cannot use "host" for the field name because LS is using as JSON. If you still want to use it, addd: mutate{ remove_field => ["host"] above grok.

[ameeto17 (Ameet Castell)]

Thanks so much for this Rios. and for the detailed explanantion.

Yes you are right. I have logs that contain version and some that dont. Hence wanted to use multiple pattern matches to match all those conditions. Your example above is exactly what i needed.

I am using multiple pattern matches like that

"%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:xxx_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software} Software Version %{DATA:rel_version} %{WORD:sha}",
          "%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:xxx_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software}%{SPACE}Software%{DATA:rel_version} %{WORD:sha}",
          "%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:xxx_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software} %{DATA:rel_version} %{WORD:sha}"

Will go ahead and add the mutate part for the second match above where rel_Version could be blank

[Rios]

No need for multiple match, just use the optional fields.

[ameeto17 (Ameet Castell)]

okies. sounds good. will try it out

[ameeto17 (Ameet Castell)]

@Rios

if (![version]){
   mutate {  add_field => {"version" => "1.0.1"} }
   }

im wondering if i should use this instead of add field since version would get created anyways with a blank value. so instead of adding just update?

if (![version]){
   mutate {  update => {"version" => "1.0.1"} }
   }

[ameeto17 (Ameet Castell)]

right now my final log looks like that with rel_veersion (field name i have used is rel_version) showing up with -

I want that to have 1.0.1

pattern i used was

%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:keno_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software}%{SPACE}Software%{DATA:rel_version} %{WORD:sha}"

that works perfectly to match this log pattern but need the rel_version above to have 1.0.1
Dec 14 03:13:01 ppddc1kfep302 keno-checker: Context SHA of VSP Logger Software da39a3ee5e6b4b0d3255bfef95601890afd80709

[Rios]

Use improved grok to handle both cases:

   grok {
     match => { "message" => "%{SYSLOGTIMESTAMP}%{SPACE}%{IPORHOST:my_host}%{SPACE}%{DATA}:%{SPACE}Context SHA of %{DATA:software} Software (%{WORD})?( %{DATA:version} )?%{WORD:hash}$" }
   }

You cannot use update since field do not exist in case when is not created by grok.
The mark - means field do not exist.

[ameeto17 (Ameet Castell)]

@Rios

add_field

ok awesome. thanks once again. much appreciated.