I'm trying to create a multi-line postfix grok pattern to have messageid, userid, username, queueid, from,to,size,etc. in the same document for graphing in kibana.
is there a way to match elements in the middle of messages without having to match everything before it?
"%{SYSLOGTIMESTAMP:pf_ts} %{HOSTNAME:hostname} %{NOTSPACE:pf_program}/%{NOTSPACE}: %{NOTSPACE:queueid}: uid=%{NUMBER:uid} from=<%{USERNAME:username}>",
%{SYSLOGTIMESTAMP} %{HOSTNAME} %{NOTSPACE} %{POSTFIX_QUEUEID}: message-id=<%{NOTSPACE:messageid}>",
....
instead of
%{SYSLOGTIMESTAMP:postfix_ts} %{HOSTNAME:postfix_hostname} %{NOTSPACE:postfix_program}/%{NOTSPACE}[%{NUMBER}]: %{POSTFIX_QUEUEID:postfix_queueid}: uid=%{NUMBER:postfix_uid} from=<%{NOTSPACE:postfix_from_user}> %{SYSLOGTIMESTAMP} %{HOSTNAME} %{NOTSPACE} %{POSTFIX_QUEUEID}: message-id=<%{NOTSPACE:postfix_messageid}> %{SYSLOGTIMESTAMP} %{HOSTNAME} %{NOTSPACE} %{POSTFIX_QUEUEID}: from=<%{NOTSPACE:postfix_from}>, size=%{NUMBER:postfix_size}, nrcpt=%{NUMBER:postfix_nrcpt} (queue active) %{SYSLOGTIMESTAMP} %{HOSTNAME} %{NOTSPACE} %{POSTFIX_QUEUEID}: to=<%{NOTSPACE:postfix_to}>, orig_to=<%{NOTSPACE:postfix_orig_to}>, relay=%{WORD:postfix_relay}, delay=%{NUMBER:postfix_delay}, delays=%{NOTSPACE:postfix_delays}, dsn=%{NOTSPACE:postfix_dsn}, status=%{NOTSPACE:postfix_status}
example log:
"Sep 30 17:30:23 rory-dev postfix/pickup[21766]: CEDD7F80708: uid=0 from= Sep 30 17:30:23 rory-dev postfix/cleanup[24115]: CEDD7F80708: message-id=20151001003023.CEDD7F80708@rory-dev.localdomain Sep 30 17:30:23 rory-dev postfix/qmgr[2079]: CEDD7F80708: from=root@rory-dev.localdomain, size=308, nrcpt=1 (queue active) Sep 30 17:30:23 rory-dev postfix/local[24278]: CEDD7F80708: to=root@localhost.localdomain, orig_to=root@localhost, relay=local, delay=0.11, delays=0.07/0.01/0/0.03, dsn=2.0.0, status=sent (delivered to mailbox) Sep 30 17:30:23 rory-dev postfix/qmgr[2079]: CEDD7F80708: removed",
Thank