Logstash's grok GREEDYDATA pattern takes word before it starts

Hello everyone,

I've been modifying GROK patterns for postfix from GitHub - ActionScripted/elastic-kibana-postfix: Kibana dashboards, visualizations and searches for Postfix so that it can include "warning" from postfix' header_check since I've added subject to appear in log file.
I've also modified rsyslog so that log file from postfix' header_check goes into separate file.
Basically i wanted specific "postfix_to" and "postfix_from" with "subject" field. Since there is no 'subject' field i just wanted it to go to postfix_message, so there is no need to chase it through logs. But. There is a catch.

Example from log file is like this:

6F4234C1362: warning: header Subject: Embedded Net DVR: Motion Detected On Channel A3 from mail.xxx.rs[77.66.77.66]; from=from@mail.rs to=another@recipient.com proto=ESMTP helo=<mail.xxx.rs>

I've setup a pattern like this:

SEPARATOR from ([a-zA-Z0-9.]{3,}[[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}]:wink:
POSTFIX_WARNING_WITH_SUBJECT (%{POSTFIX_QUEUEID:postfix_queueid}: )?warning: header %{GREEDYDATA:postfix_message} %{SEPARATOR} from=<%{DATA:postfix_from}> to=<%{DATA:postfix_to}> proto=.*

And this pattern checks out JUST FINE in Dev Tools/Grok Debugger. But when live..
All is good except postfix_message which should contain JUST the subject with Subject prefix... But instead, it contains a word before GREEDYDATA and a word AFTER.

This is from Grok Debugger:

{
"postfix_to": "another@recipient.com",
"postfix_queueid": "6F4234C1362",
"postfix_subject": "Subject: Embedded Net DVR: Motion Detected On Channel A3 ",
"postfix_from": "from@mail.rs"
}

I love this ! This is cool!
But live i get this

"postfix_message": " header Subject: Embedded Net DVR: Motion Detected On Channel A3 from mail.3dnet.rs[77.105.38.87]"

to, from and queueid are fine.
Why is that so ?

You have not given us the definitons of all the patterns you are using, so it is hard to tell.

These are the patterns

I've added only SEPARATOR and POSTFIX_WARNING_WITH_SUBJECT in same file.
And edited:

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}

to

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}|%{POSTFIX_WARNING_WITH_SUBJECT}

However, I've noticed that if i move POSTFIX_WARNING_WITH_SUBJECT to be the first one:

POSTFIX_WARNING %{POSTFIX_WARNING_WITH_SUBJECT}|%{POSTFIX_WARNING_WITH_KV}|%{POSTFIX_WARNING_WITHOUT_KV}

Now everything is OK! :slight_smile: I just can't figure out why. My pattern is (should be) more precise. Since it includes to and from.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.