Grok for email log

Elastic Stack Logstash
4

I assume that you consume the entire log as a single event using multiline codec. If you are consuming it one line at a time things will be a little different. Decide which headers you want to parse and use the pattern I described to extract them. For example

    grok {
        break_on_match => false
        match => {
            message => [
                "^Date: (?<Date>[^
]*)
",
                "^Subject: (?<Subject>[^
]*)
",
                "^X-Spam-Level: (?<X-Spam-Level>[^
]*)
",
                "^X-Spam-Status: (?<X-Spam-Status>[^
]*)
"
            ]
        }
    }

Once you have extracted them you may require additional parsing to extract fields from those headers.

For Received you cannot do it using grok because there are multiple occurences, however, it is simple to do a similar regex match in ruby

    ruby {
        code => 'event.set("Received", event.get("message").scan(/^Received: ([^
]+
)/).flatten)'
    }

That will set Received to an array of 5 strings which contain the contents of the Received headers.