Help, Cannot get multiline to work correctly

Elastic Stack Beats
1

I have this raw log output:

[2020-08-07T12:45:33+02:00] 10.8.0.84 ERROR Exception: <log:context>{"exception":"[object] (UnexpectedValueException(code: 0): This is to simulate uncaught exceptions at /home/app1/products/services/lvs-api/releases/2.3.5/src/App/Action/StatusAction.php:29)
[stacktrace]
#0 /home/app1/products/services/lvs-api/releases/2.3.5/vendor/zendframework/zend-expressive/src/Middleware/LazyLoadingMiddleware.php(80): App\\Action\\StatusAction->__invoke(Object(Zend\\Diactoros\\ServerRequest), Object(Zend\\Diactoros\\Response), Object(Closure))
#1 /home/app1/products/services/lvs-api/releases/2.3.5/vendor/zendframework/zend-expressive/src/Middleware/DispatchMiddleware.php(90): Zend\\Expressive\\Middleware\\LazyLoadingMiddleware->process(Object(Zend\\Diactoros\\ServerRequest), Object(Zend\\Stratigility\\Next))

This is the multiline pattern that I use to trap the first line of the message:

^\[\d+|^\d{4}\/\d{2}\/\d{2}

These are the settings for multiline in filebeat:

  multiline.pattern: ^\[\d+|^\d{4}\/\d{2}\/\d{2} 
  multiline.negate: True
  multiline.match: after

According to my regex it will trap the line that starts with:

[2020

What is actually happening:

The message is being split up into multiple messages like so:

log message-1: [2020-08-07T12:45:33+02:00] 10.8.0.84 E
log message-2: [stacktrace]
log message-3: #0 /home/app1/pr
log message-4: #1 /home/app1/pr

json output also confirms this.

Can anyone advise how I can fix this?

2

This worked for me for the same log format..

multiline.pattern: ^[[0-9]{2,4}-[0-9]{1,2}-[0-9]{1,2}
multiline.negate: True
multiline.match: after

Regards,
Sai

3

Thanks Sai, I will give this a try.

4

Not sure whats going on, but I applied your suggestion. When I view the output in Kibana the messages are still split.

5

Actually I looked at the regex again, shouldn't the first character be escaped [

6

Indeed I somehow missed adding / :sweat_smile:.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.