Optional match for a grok pattern

[moep]

Hey community,

I would like to build an optional match for the following logline:

2023-01-11 00:00:11 1pEoP9-000LLu-Gz <= noreply@domain.de H=fqdn.domain.de (FQDN) [10.1.1.1] P=esmtpa A=login_virtual_exim:mtaspooler@domain.de S=19312 

I'm using the following grok pattern to match, but it doesnt match at the beginn of A= in the logline.

(%{EXIM_DATE}) (%{EXIM_MSGID}) (%{EXIM_FLAGS}) ?(%{EMAILADDRESS:exim_sender}) ?(%{EXIM_REMOTE_HOST}) ?(%{EXIM_PROTOCOL}) ?(X=(?<exim_tls>TLS1.[0-9]):(%{NOTSPACE:exim_ciphers})) ?(CV=%{WORD:exim_cv_value}) ?(A=%{NOTSPACE:exim_mtaspooler}) ?(%{EXIM_MSG_SIZE}) ?(%{EXIM_HEADER_ID})

[Badger]

@moep

I'm using the following grok pattern to match, but it doesnt match at the beginn of A= in the logline.

That is because the CV= field is required, not optional. I suggest making all those fields optional (assuming that they are) and pulling the ? inside the sub-patterns.

 grok { match => { "message" => "(%{EXIM_DATE}) (%{EXIM_MSGID}) (%{EXIM_FLAGS}) ?(%{EMAILADDRESS:exim_sender}) ?(%{EXIM_REMOTE_HOST}) ?(%{EXIM_PROTOCOL})( X=(?<exim_tls>TLS1.[0-9]):(%{NOTSPACE:exim_ciphers}))?( CV=%{WORD:exim_cv_value})?( A=%{NOTSPACE:exim_mtaspooler})?( %{EXIM_MSG_SIZE})?( %{EXIM_HEADER_ID})?" } }

[moep]

so basicly you removed the ? outside, so its optional changed the whitespace in the brackets?

[Badger]

I changed

 ?(CV=%{WORD:exim_cv_value})

to

( CV=%{WORD:exim_cv_value})?

and likewise for the other fields.