Logstash's grok GREEDYDATA pattern takes word before it starts

Elastic Stack Logstash
1

Hello everyone,

I've been modifying GROK patterns for postfix from GitHub - ActionScripted/elastic-kibana-postfix: Kibana dashboards, visualizations and searches for Postfix so that it can include "warning" from postfix' header_check since I've added subject to appear in log file.
I've also modified rsyslog so that log file from postfix' header_check goes into separate file.
Basically i wanted specific "postfix_to" and "postfix_from" with "subject" field. Since there is no 'subject' field i just wanted it to go to postfix_message, so there is no need to chase it through logs. But. There is a catch.

Example from log file is like this:

6F4234C1362: warning: header Subject: Embedded Net DVR: Motion Detected On Channel A3 from mail.xxx.rs[77.66.77.66]; from=from@mail.rs to=another@recipient.com proto=ESMTP helo=<mail.xxx.rs>

I've setup a pattern like this:

SEPARATOR from ([a-zA-Z0-9.]{3,}[[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}]:wink:
POSTFIX_WARNING_WITH_SUBJECT (%{POSTFIX_QUEUEID:postfix_queueid}: )?warning: header %{GREEDYDATA:postfix_message} %{SEPARATOR} from=<%{DATA:postfix_from}> to=<%{DATA:postfix_to}> proto=.*

And this pattern checks out JUST FINE in Dev Tools/Grok Debugger. But when live..
All is good except postfix_message which should contain JUST the subject with Subject prefix... But instead, it contains a word before GREEDYDATA and a word AFTER.

This is from Grok Debugger:

{
"postfix_to": "another@recipient.com",
"postfix_queueid": "6F4234C1362",
"postfix_subject": "Subject: Embedded Net DVR: Motion Detected On Channel A3 ",
"postfix_from": "from@mail.rs"
}

I love this ! This is cool!
But live i get this

"postfix_message": " header Subject: Embedded Net DVR: Motion Detected On Channel A3 from mail.3dnet.rs[77.105.38.87]"

to, from and queueid are fine.
Why is that so ?

2

You have not given us the definitons of all the patterns you are using, so it is hard to tell.

3

These are the patterns

I've added only SEPARATOR and POSTFIX_WARNING_WITH_SUBJECT in same file.
And edited:

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}

to

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}|%{POSTFIX_WARNING_WITH_SUBJECT}

However, I've noticed that if i move POSTFIX_WARNING_WITH_SUBJECT to be the first one:

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_SUBJECT}|%{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}

Now everything is OK! :slight_smile: I just can't figure out why. My pattern is (should be) more precise. Since it includes to and from.