FortiMail logs are being combined in TCP input

I suspect that the problem is FortiMail is sending RFC 5425 compliant "octet-counted" messages (i.e. no trailing newline). See this thread for a little more detail. Not sure if FortiMail can be configured to use traditional format instead of RFC 5425.

The existing tcp input is line-oriented, but you need a byte-oriented input. The difference between the two creates issues that were long ago recognized. (Note that the issue is still open, none of the re-architecture that was envisioned actually happened.)

You could re-write the tcp input to be byte oriented, but it would be much simpler to add a syslog server that can read RFC 5425 compliant messages from FortiMail and forward them to logstash in traditional newline separated format.

Your problem here is that if a CSV field contains double quotes then the entire field must be quoted. So instead of

session_id="33AKq7LU018089-33AKq7LX018089"

it would have to be

"session_id=""33AKq7LU018089-33AKq7LX018089"""
2 Likes