# FILEBEAT REGULAR EXPRESSION FAILING IN MULTI LINE PATTERN

**URL:** <https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 25, 2019, 3:05pm UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494 "2019-04-25T15:05:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bomba](https://avatars.discourse-cdn.com/v4/letter/b/df705f/32.png) [@bomba](https://discuss.elastic.co/u/bomba)\
**Post date:** [April 25, 2019, 3:05pm UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494/1 "2019-04-25T15:05:10Z")

</div>

We are using multiline pattern for file beats to parse the application logs.  
Multiline patterns is not able to merge related line as one log

PS : We are using [https://play.golang.org/](https://play.golang.org/) for testing.  
We are not able to find any regex for negative lookups

Regular Expression :

1. ^[[:space:]]
2. ^[[:space:]]+(at|.{3})\b|^Traceback (most recent call last):
3. ^[[:space:]]|^Traceback|^exception

**Multiline Config :**  
multiline.negate: false  
multiline.match: after

Expected Output :  
Matched Pattern 1:  
ERROR 2019-04-18 12:04:11,193 publisher\_viewsets 16 140543468234496 (2, 'XYZXYZ\_ERR\_RECORD\_NOT\_FOUND', 'src/XXXXXXXXXXXXXXX', 113)  
Traceback (most recent call last):  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 131, in XYZXYZ  
user\_bins = XYZXYZ\_ERR\_RECORD\_NOT\_FOUND.XYZXYZ(XYZXYZ)  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 203, in XYZXYZ  
raise e  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 199, in XYZXYZ  
key, meta, bin = self.XYZ.get(key)  
exception.RecordNotFound: (2, 'XYZXYZ\_ERR\_RECORD\_NOT\_FOUND', 'src/XYZ/XYZ/XYZ.c', 113)

Matched Pattern 2:  
ERROR 2019-04-19 11:38:48,148 event\_viewsets 16 140543468234496 Signature has expired  
Traceback (most recent call last):  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 131, in XYZXYZ  
user\_bins = XYZXYZ\_ERR\_RECORD\_NOT\_FOUND.XYZXYZ(XYZXYZ)  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 203, in XYZXYZ  
raise e  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 199, in XYZXYZ  
key, meta, bin = self.XYZ.get(key)  
File "/usr//opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 284, in verify  
'signatures' + repr(self.verifylog))  
xyz.xyz.XYZ: Verification failed for all XYZ['Failed: [Invalid("Verification failed {XYZ('Temporary Text.',)}",)]']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/views/event\_viewsets.py", line 269, in track  
client\_cookie, is\_new, \_ = XYZ\_or\_XYZ\_XYZ(request, UID\_COOKIE\_NAME)  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/utils.py", line 164, in XYZ\_or\_XYZ\_XYZ  
XYZ\_or\_XYZ\_XYZ = ClientCookie.get\_decoded\_cookie(arbitrary\_cookie).XYZ\_or\_XYZ\_XYZ  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/utils.py", line 69, in get\_decoded\_cookie  
payload = jwt\_decode\_handler(encoded\_cookie)  
File "/usr/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/api.xx", line 175, in \_XYZ\_exp  
raise ExpiredSignatureError('XYZ has XYZ')  
XYZ.XYZ.ExpiredSignatureError: XYZ XYZ XYZ

Matched Pattern 3:  
DEBUG 2019-04-18 12:04:11,706 event\_viewsets 17 140543334016768 Returning response with client cookie:XYZ

Matched Pattern 4:  
WARNING 2019-04-18 10:39:26,416 base 19 140544438826752 Not Found: /XYZ/XYZ.exp

**Sample Log :**  
\  
ERROR 2019-04-18 12:04:11,193 publisher\_viewsets 16 140543468234496 (2, 'XYZXYZ\_ERR\_RECORD\_NOT\_FOUND', 'src/XXXXXXXXXXXXXXX', 113)  
Traceback (most recent call last):  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 131, in XYZXYZ  
user\_bins = XYZXYZ\_ERR\_RECORD\_NOT\_FOUND.XYZXYZ(XYZXYZ)  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 203, in XYZXYZ  
raise e  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 199, in XYZXYZ  
key, meta, bin = self.XYZ.get(key)  
exception.RecordNotFound: (2, 'XYZXYZ\_ERR\_RECORD\_NOT\_FOUND', 'src/XYZ/XYZ/XYZ.c', 113)  
DEBUG 2019-04-18 12:04:11,701 decorators 17 140543334016768 Request Body:{}  
DEBUG 2019-04-18 12:04:11,701 event\_viewsets 17 140543334016768 Site ID not present as query parameter, doing lookup in XYZ\_MAP ...  
DEBUG 2019-04-18 12:04:11,706 event\_viewsets 17 140543334016768 Returning response with client cookie:XYZ  
WARNING 2019-04-18 10:39:26,416 base 19 140544438826752 Not Found: /XYZ/XYZ.exp  
ERROR 2019-04-19 11:38:48,148 event\_viewsets 16 140543468234496 Signature has expired  
Traceback (most recent call last):  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 131, in XYZXYZ  
user\_bins = XYZXYZ\_ERR\_RECORD\_NOT\_FOUND.XYZXYZ(XYZXYZ)  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 203, in XYZXYZ  
raise e  
File "/opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 199, in XYZXYZ  
key, meta, bin = self.XYZ.get(key)  
File "/usr//opt/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX/", line 284, in verify  
'signatures' + repr(self.verifylog))  
xyz.xyz.XYZ: Verification failed for all XYZ['Failed: [Invalid("Verification failed {XYZ('Temporary Text.',)}",)]']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/views/event\_viewsets.py", line 269, in track  
client\_cookie, is\_new, \_ = XYZ\_or\_XYZ\_XYZ(request, UID\_COOKIE\_NAME)  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/utils.py", line 164, in XYZ\_or\_XYZ\_XYZ  
XYZ\_or\_XYZ\_XYZ = ClientCookie.get\_decoded\_cookie(arbitrary\_cookie).XYZ\_or\_XYZ\_XYZ  
File "/opt/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/utils.py", line 69, in get\_decoded\_cookie  
payload = jwt\_decode\_handler(encoded\_cookie)  
File "/usr/XXXXXXXXX/XXXXXXXXX/XXXXXXXXX/api.xx", line 175, in \_XYZ\_exp  
raise ExpiredSignatureError('XYZ has XYZ')  
XYZ.XYZ.ExpiredSignatureError: XYZ XYZ XYZ  
WARNING 2019-04-19 11:40:52,260 base 17 140543459579648 Not Found: /favicon.ico  
DEBUG 2019-04-19 11:45:40,707 publisher\_viewsets 17 140543435130624 Getting deal record for subdomain:XYZ.com  
\\

---

<div class="post-metadata">

**Author:** ![bomba](https://avatars.discourse-cdn.com/v4/letter/b/df705f/32.png) [@bomba](https://discuss.elastic.co/u/bomba)\
**Post date:** [April 26, 2019, 3:29pm UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494/2 "2019-04-26T15:29:00Z")

</div>

Any suggestions

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [April 29, 2019, 9:27am UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494/3 "2019-04-29T09:27:47Z")

</div>

Could you please share your filebeat configuration you are testing with? Also, please use `</>` when sharing your logs, as simply pasting it here, does not preserve whitespaces.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 27, 2019, 9:27am UTC](https://discuss.elastic.co/t/filebeat-regular-expression-failing-in-multi-line-pattern/178494/4 "2019-05-27T09:27:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.