Hi,
I am delivering logs from Postfix to Logstash and I am displaying them on a Kibana dashboard to find out not-delivered or bounced e-mails.
The problem is that Postfix inserts important information about the message on multiple lines:
1 Oct 9 18:19:58 mailserver postfix/smtpd[11513]: 7958440AA2: client=client.fqdn[123.123.123.123]
2 Oct 9 18:19:59 mailserver postfix/cleanup[11518]: 7958440AA2: message-id=<>
3 Oct 9 18:19:59 mailserver postfix/qmgr[26050]: 7958440AA2: from=<user@client.fqdn>, size=841, nrcpt=1 (queue active)
4 Oct 9 18:19:59 mailserver postfix/smtpd[11513]: disconnect from client.fqdn[123.123.123.123] helo=1 mail=1 rcpt=1 data=1 quit=1 commands=5
5 Oct 9 18:20:10 mailserver postfix/smtp[11519]: 7958440AA2: to=<recipient@server.fqdn>, relay=relay.fqdn[111.111.111.111]:25, delay=12, delays=1/0.01/10/1.3, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 1F2BF9679C4)
6 Oct 9 18:20:10 mailserver postfix/qmgr[26050]: 7958440AA2: removed
In this example, the sender of the message is displayed into the log at line 3, and message delivery information (status, remote queue ID, etc.) is displayed into the log at line 5.
As these logs are sharing the message ID (7958440AA2 in this example) I wish to find a way to add the from metadata from the line 3 to the log on the line 5 (which is parsed from Kibana).
I found the Aggregate filter plugin at https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html, and I am looking for some hint on how to use it to solve my problem.
Could you help me please?
Thanks!