Hello everyone,
We moved from very old version of ELK stack to the 5.0 and the multiline config only takes the first one, even though the test on go playground seems OK.
We parse OpenAS2 logs, where we added the tag <msglog> at the beginning of each line.
When we send a file with it, the recipient send MDN back, which is multiline log. We only have the first line stored in elasticsearch.
The configuration :
# OpenAS2 - input_type: log paths: - /opt/openas2server/logs/*.log multiline: pattern: '^<msglog>.*' negate: true match: before fields: type: as2
Our typical log 
<msglog>2016-11-07 10:11:48,942 | INFO | Timer-2 | .receiver.DirectoryPollingModule ? | processing /opt/openas2server/bin/../config/../data/__FILE__</msglog>
<msglog>2016-11-07 10:11:48,942 | INFO | Timer-2 | or.receiver.MessageBuilderModule ? | file assigned to message /opt/openas2server/bin/../config/../data/__FILE__ [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:48,942 | INFO | Timer-2 | processor.sender.AS2SenderModule ? | message sender invoked [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:48,958 | INFO | Timer-2 | processor.sender.AS2SenderModule ? | Save Original mic & message id information into file: /opt/openas2server/bin/../config/../data/pendinginfoMDN3/__AS2_ID__ [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:48,958 | INFO | Timer-2 | processor.sender.AS2SenderModule ? | Connecting to: http://__IP__:__PORT__ [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:48,970 | INFO | Timer-2 | processor.sender.AS2SenderModule ? | transferred 2829 bytes in 0.12 seconds at 230.230 KBps [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:49,001 | INFO | Timer-2 | org.openas2.util.AS2Util ? | received MDN [automatic-action/MDN-sent-automatically; processed] [<__AS2_ID__>]</msglog>
<msglog>2016-11-07 10:11:49,001 | INFO | Timer-2 | org.openas2.util.AS2Util ? | Message From:{as2_id=__FROM_ID__, name=__FROM_NAME__, email=__FROM_EMAIL__, x509_alias=__FROM_ALIAS__}To:{as2_id=__TO_ID__, name=__TO_NAME__, email=__TO_EMAIL__, x509_alias=__TO_ALIAS__}
Headers:{Message-Id=<__AS2_ID__>, Subject=From QALTRT01 to Kontur_Ru, Content-Type=application/octet-stream, Content-Disposition=Attachment; filename="__FILE__"}
Attributes:{pendinginfo=/opt/openas2server/bin/../config/../data/pendinginfoMDN3/__AS2_ID__, errordir=/opt/openas2server/bin/../config/../data/toKontur_Ru/error, filename=__FILE__, destination_ip=__DEST_IP__, filepath=/opt/openas2server/bin/../config/../data/__FILE__, destination_port=__PORT__, pendingfilename=/opt/openas2server/bin/../config/../data/pendingMDN3/__MDN__, status=pending}
MDN:MDN From:{as2_id=__FROM_ID__, name=__FROM_NAME__, email=__FROM_EMAIL__, x509_alias=__FROM_ALIAS__}To:{as2_id=__TO_ID__, name=__TO_NAME__, email=__TO_EMAIL__, x509_alias=__TO_ALIAS__}
Headers:{Date=Mon, 07 Nov 2016 10:14:04 +0200, From=null, Message-Id=<__MDN_ID__>, Subject=Your Requested MDN Response, MIME-Version=1.0, Content-Type=multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="----=_Part_59_254024475.1478506444407", AS2-To=__TO_AS2__, AS2-From=__FROM_AS2__, Server=OpenAS2 v2.0.0, AS2-Version=1.1, Content-Length=2255}
Attributes:{FINAL_RECIPIENT=rfc822; __AS2_ID__ IC=1fm1uN1IA6grklYr0Hcr84hAQC21bBTBUNnTyPdtmn8=, SHA256, REPORTING_UA=OpenAS2 v2.0.0@/__IP__:__PORT__, ORIGINAL_MESSAGE_ID=<__AS2_ID__>, ORIGINAL_RECIPIENT=rfc822; __AS2_ID__, DISPOSITION=automatic-action/MDN-sent-automatically; processed}
Text:
The message sent to Recipient __AS2_ID__ on Mon, 07 Nov 2016 10:11:48 +0200 with Subject From __FROM__ to __TO__ has been received, the EDI Interchange was successfully decrypted and it's integrity was verified. In addition, the sender of the message, Sender __SENDER__ at Location /__IP__ was authenticated as the originator of the message. There is no guarantee however that the EDI Interchange was syntactically correct, or was received by the EDI application/translator.
</msglog>
<msglog>2016-11-07 10:11:49,002 | INFO | Timer-2 | org.openas2.util.AS2Util ? | deleted /opt/openas2server/bin/../config/../data/pendingMDN3/__MDN_FILE__ [<__AS2_ID__>]</msglog>
On go playground, we see that every line starting with <msglog> is tagged with false, every other line is tagged with true.
So does filebeat wrapped multiline logs into one before sending to logstash ?
Or does logstash needs a configuration to handle these multilines ? We will also handle java stack trace later, so we will have different multiline logs.
regards.