# Filebeat multiline regular expression

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-regular-expression/201371>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [September 27, 2019, 9:38am UTC](https://discuss.elastic.co/t/filebeat-multiline-regular-expression/201371 "2019-09-27T09:38:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mahipavan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahipavan/32/54973_2.png) [@Mahipavan](https://discuss.elastic.co/u/Mahipavan)\
**Post date:** [September 27, 2019, 9:38am UTC](https://discuss.elastic.co/t/filebeat-multiline-regular-expression/201371/1 "2019-09-27T09:38:36Z")

</div>

Hi,

I am trying to implement filebeat multiline pattern on our custom application logs which contains entries of different formats as shown below.

[2019-09-27T10:35:44.257+02:00] [osb\_server1] [NOTIFICATION] [oracle.osb.logging.pipeline] [tid: 65] [userId:] [ecid: db6c52be-d674-418a-bb79-b6e5aa5ac6a8-00075673,0] [APP: Service Bus Logging] [FlowId: 0000Mplr9363JBl6wvvX6G1TZDkH0006UW] [GetMessageFromTakeCargoQueue, GetMessageFromTakeCargoQueue\_request, LoopForRegisteredAskoCompainesInTakeCargo, REQUEST] \<\<\<\<\<\< No of deleted Messages \>\>\>\>\>\>: 0  
[2019-09-27T10:35:44.262+02:00] [osb\_server1] [NOTIFICATION] [oracle.osb.logging.pipeline] [tid: 65] [userId:] [ecid: db6c52be-d674-418a-bb79-b6e5aa5ac6a8-00075673,0] [APP: Service Bus Logging] [FlowId: 0000Mplr9363JBl6wvvX6G1TZDkH0006UW] [CSF\_TrackMessage\_To\_Elastic, request-N53ed173e.N32fad4e5.0.16871088aa4.N7fc4, Log, REQUEST] \<ns0:trackMessage xmlns:WL5G3N1="urn:company.no/Services/AnyXMLService" xmlns:SOAP-ENV="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" xmlns:ns0="[http://xmlns.oracle.com/TrackingDetails](http://xmlns.oracle.com/TrackingDetails)"\>[[  
ns0:header  
ns0:isDetailsRequired  
ns0:isDetailsRequiredY\</ns0:isDetailsRequired\>  
ns0:queryDetailsBy1\</ns0:queryDetailsBy\>  
\</ns0:isDetailsRequired\>  
\</ns0:header\>  
\</ns0:trackMessage\>  
]]  
[2019-09-27T11:03:44.608+02:00] [osb\_server1] [NOTIFICATION] [oracle.osb.logging.pipeline] [tid: 106] [userId:] [ecid: db6c52be-d674-418a-bb79-b6e5aa5ac6a8-00077478,0] [APP: Service Bus Logging] [FlowId: 0000MplxZRu3JBl6wvvX6G1TZDkH0006ZV] [GetMessageFromTakeCargoQueue, GetMessageFromTakeCargoQueue\_request, LoopForRegisteredAskoCompainesInTakeCargo, REQUEST] \<\<\<\<\<\< No of deleted Messages \>\>\>\>\>\>: 0  
[2019-09-27T11:03:44.671+02:00] [osb\_server1] [NOTIFICATION] [oracle.osb.logging.pipeline] [tid: 106] [userId:] [ecid: db6c52be-d674-418a-bb79-b6e5aa5ac6a8-00077478,0] [APP: Service Bus Logging] [FlowId: 0000MplxZRu3JBl6wvvX6G1TZDkH0006ZV] [GetMessageFromTakeCargoQueue, GetMessageFromTakeCargoQueue\_request, LoopForRegisteredAskoCompainesInTakeCargo, REQUEST] \<\<\<\<\<\< No of deleted Messages \>\>\>\>\>\>: 0  
[2019-09-27T11:03:44.916+02:00] [osb\_server1] [NOTIFICATION] [oracle.osb.logging.pipeline] [tid: 106] [userId:] [ecid: db6c52be-d674-418a-bb79-b6e5aa5ac6a8-00077478,0] [APP: Service Bus Logging] [FlowId: 0000MplxZRu3JBl6wvvX6G1TZDkH0006ZV] [CSF\_TrackMessage\_To\_Elastic, request-N53ed173e.N32fad4e5.0.16871088aa4.N7fc4, Log, REQUEST] \<ns0:trackMessage xmlns:WL5G3N1="urn:company.no/Services/AnyXMLService" xmlns:SOAP-ENV="[http://schemas.xmlsoap.org/soap/envelope/](http://schemas.xmlsoap.org/soap/envelope/)" xmlns:ns0="[http://xmlns.oracle.com/TrackingDetails](http://xmlns.oracle.com/TrackingDetails)"\>[[  
ns0:header  
ns0:isDetailsRequired  
ns0:isDetailsRequiredY\</ns0:isDetailsRequired\>  
ns0:queryDetailsBy1\</ns0:queryDetailsBy\>  
\</ns0:isDetailsRequired\>  
\</ns0:header\>  
\</ns0:trackMessage\>  
]]

I want filbeat read only lines which start with timestamp,contains "To\_Elastic" and append all the following lines until it encounters "]]".

I am trying with the below multiline configuration but this also picks all those lines which doesn't contain "To\_Elastic" and clubs them into a single event.

multiline.pattern: '^[[0-9]{4}-[0-9]{2}-[0-9]{2}._To\_Elastic._'  
multiline.max\_lines: 50000  
multiline.flush\_pattern: '^]]'  
multiline.timeout: 100  
multiline.negate: true  
multiline.match: after

Requesting help in fixing this. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 25, 2019, 9:38am UTC](https://discuss.elastic.co/t/filebeat-multiline-regular-expression/201371/2 "2019-10-25T09:38:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.