More easily track a single email between services and servers

Elastic Stack Kibana
1

Hi all, We are sending our postfix logs to elasticsearch using journalbeat. Our email system sends messages from one server to another and on a given server, from one service to another service (spam checking etc). Each time it gets a new message id.

If I want to know what happened to a single email message, I open up Kibana Discover and find the first message id. Then I have to keep altering my search parameters to trace it from one place to the next. All of this works and is more convenient then logging into multiple servers and grepping the logs.

But... I was wonering if there is an easier way - any built in function that knows how to discover the next message id from the current one so I could just plug in one message id and see the entire trail?

2

Hi
Quick question, that message id is generated by postfix? is the initial message id also part of the messages when it arrives at other services? What's the structure of the document when it's in ingested in Elasticsearch?

Thx!
Best,
Matthias

3

Thanks for the follow up. Yes, the message id is generated by postfix and it is cleanly passed on for each step. Here's an example.

  • It starts on server foo with message id EF2DC3F27
  • Then it gets passed on to server bar with message id A76481DCFF
  • Then it gets passed on to the next server with message id: DD20F405B5
    Feb 03 12:22:27 foo postfix/smtp[4508]: EF2DC3F27: to=<xxxxx@lists.xxxxx.xxx>, relay=xxx.xxx.xxx[xxx.xxx.xxx.xxx]:25, delay=0.89, delays=0.49/0.01/0.13/0.25, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as A76481DCFF)
    Feb 04 09:42:10 bar postfix/smtpd[6085]: A76481DCFF: client=foo[xx.xx.xx.xx]
    Feb 04 09:42:10 bar postfix/cleanup[6145]: A76481DCFF: message-id=<CAFY6a4nyw-UEwyM5zOfvaJExZrz4H+KoROC6rJ8++fVc2RdDLA@mail.gmail.com>
    Feb 04 09:42:10 bar postfix/qmgr[14059]: A76481DCFF: from=<xxxx@xxxx.xxx.org>, size=22152, nrcpt=1 (queue active)
    Feb 04 09:42:10 bar postfix/smtp[6144]: A76481DCFF: to=<xxx@xxx.xxx.org>, relay=xxx.xxx.xxx[xxx.xxx.xxx.xxx]:25, delay=0.23, delays=0.2/0/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as DD20F405B5)
    Feb 04 09:42:10 bar postfix/qmgr[14059]: A76481DCFF: removed

At the moment, the entire text is in one field, but I could use dissect to break out the message id for every line that has one.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.