# Multiline pattern & flush pattern in filebeat
**URL:** https://discuss.elastic.co/t/multiline-pattern-flush-pattern-in-filebeat/159885
**Category:** Beats
**Tags:** filebeat
**Created:** [December 7, 2018, 8:46am UTC](https://discuss.elastic.co/t/multiline-pattern-flush-pattern-in-filebeat/159885 "2018-12-07T08:46:35Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@vennila](https://discuss.elastic.co/u/vennila)
#### Post date: [December 7, 2018, 8:46am UTC](https://discuss.elastic.co/t/multiline-pattern-flush-pattern-in-filebeat/159885/1 "2018-12-07T08:46:36Z")
Hi All,
I am using multiline pattern within filebeat.yml to format the logs as follows,
filebeat.inputs:
- document\_type: webapp
enabled: true
paths:
- /opt/sample/app.log
multiline.pattern: '^., [[^]]+]'
multiline.negate: true
multiline.match: after
tags: ["shop"]
exclude\_lines: ["^$"]
fields: {application: shop}
fields\_under\_root: true
tail\_files: true
My logs are having some unique ID to identify the block of logs to combine multiline logs. So I tried with flush pattern to do this in filebeat. But my logs are getting collapsed with another blocks since I am not able to use the same group value ( Unique-ID) from multiline.pattern to flush pattern.
for example, ( Only for example and this patterns are not valid one for my logs )
multiline.pattern: 'Started'
multiline.negate: true
multiline.match: after
multiline.flush\_pattern: 'Completed'
My sample log,
```auto
I, [2018-07-24T05:40:19.326479 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Processing by CartController#account_provision as JSON
I, [2018-07-24T05:40:19.326567 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Parameters: {"speed"=>"8.99", "tracking"=>{"google"=>{"clientId"=>"1274719564.1532410753"}, "offer"=>{"offer_code"=>"STREAM_HUB", "purchase_code"=>"", "referral_code"=>"", "buyer_pk"=>"", "order_id"=>""}, "web"=>{"user_agent"=>"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0", "referrer"=>"https://shop-stream-qa.ooma.com/cart?", "view"=>"desktop", "ip"=>"127.0.0.1", "tenant"=>"stream", "tracking_session_id"=>"ioiwo2j58ep2uu9tum3mo6zfcsas69am", "url"=>"https://shop-stream-qa.ooma.com/cart/checkout#review", "page"=>"https://shop-stream-qa.ooma.com/cart/checkout#review"}}}
I, [2018-07-24T05:40:20.916634 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Started GET "/cart/confirm" for 127.0.0.1 at 2018-07-24 05:40:21 +0000
I, [2018-07-24T05:40:21.813837 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Completed 200 OK in 1487ms (Views: 0.2ms)
I, [2018-07-24T05:40:21.918109 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Processing by CartController#confirm as HTML
I, [2018-07-24T05:40:21.924357 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/_templates.html.erb (3.3ms)
I, [2018-07-24T05:40:21.927207 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/_review_items.html.erb (2.2ms)
I, [2018-07-24T05:40:21.927628 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/confirm.html.erb within layouts/application (8.4ms)
I, [2018-07-24T05:40:21.929867 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_head.html.erb (2.0ms)
I, [2018-07-24T05:40:21.930372 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_gtm.html.erb (0.3ms)
I, [2018-07-24T05:40:21.930732 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_header.html.erb (0.2ms)
I, [2018-07-24T05:40:21.932368 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Completed 200 OK in 14ms (Views: 13.7ms)
```
Here, filebeat treating the followings are multiline events where actually it's not.
```auto
I, [2018-07-24T05:40:20.916634 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Started GET "/cart/confirm" for 127.0.0.1 at 2018-07-24 05:40:21 +0000
I, [2018-07-24T05:40:21.813837 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Completed 200 OK in 1487ms (Views: 0.2ms)
```
It should decide the multiline block with 'Request ID'. Is this possible using multiline pattern ?
For example I want to group my logs using request ID. Here Req IDs are,
`343ca189-6b84-4657-8dc0-1a93879482c6 & 43df0991-650e-4bcb-a3b2-a20625515fdd`
The blocks I need is,
```auto
I, [2018-07-24T05:40:19.326479 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Processing by CartController#account_provision as JSON
I, [2018-07-24T05:40:19.326567 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Parameters: {"speed"=>"8.99", "tracking"=>{"google"=>{"clientId"=>"1274719564.1532410753"}, "offer"=>{"offer_code"=>"STREAM_HUB", "purchase_code"=>"", "referral_code"=>"", "buyer_pk"=>"", "order_id"=>""}, "web"=>{"user_agent"=>"Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:57.0) Gecko/20100101 Firefox/57.0", "referrer"=>"https://shop-stream-qa.ooma.com/cart?", "view"=>"desktop", "ip"=>"127.0.0.1", "tenant"=>"stream", "tracking_session_id"=>"ioiwo2j58ep2uu9tum3mo6zfcsas69am", "url"=>"https://shop-stream-qa.ooma.com/cart/checkout#review", "page"=>"https://shop-stream-qa.ooma.com/cart/checkout#review"}}}
I, [2018-07-24T05:40:21.813837 #27608] INFO -- : [343ca189-6b84-4657-8dc0-1a93879482c6] Completed 200 OK in 1487ms (Views: 0.2ms)
```
and
```auto
I, [2018-07-24T05:40:20.916634 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Started GET "/cart/confirm" for 127.0.0.1 at 2018-07-24 05:40:21 +0000
I, [2018-07-24T05:40:21.918109 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Processing by CartController#confirm as HTML
I, [2018-07-24T05:40:21.924357 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/_templates.html.erb (3.3ms)
I, [2018-07-24T05:40:21.927207 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/_review_items.html.erb (2.2ms)
I, [2018-07-24T05:40:21.927628 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered cart/confirm.html.erb within layouts/application (8.4ms)
I, [2018-07-24T05:40:21.929867 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_head.html.erb (2.0ms)
I, [2018-07-24T05:40:21.930372 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_gtm.html.erb (0.3ms)
I, [2018-07-24T05:40:21.930732 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Rendered layouts/_header.html.erb (0.2ms)
I, [2018-07-24T05:40:21.932368 #27608] INFO -- : [43df0991-650e-4bcb-a3b2-a20625515fdd] Completed 200 OK in 14ms (Views: 13.7ms)
```
Is this possible in filebeat ? Can anyone please suggest.
Thanks,
Vennila K
---
### Author:  [@vennila](https://discuss.elastic.co/u/vennila)
#### Post date: [December 11, 2018, 12:10pm UTC](https://discuss.elastic.co/t/multiline-pattern-flush-pattern-in-filebeat/159885/2 "2018-12-11T12:10:40Z")
Hi all,
shall I get any update/idea on this ?
Regards,
Vennila K
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 8, 2019, 12:10pm UTC](https://discuss.elastic.co/t/multiline-pattern-flush-pattern-in-filebeat/159885/3 "2019-01-08T12:10:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.