File beat Multiline issue

Elastic Stack Beats
1

Hello! I am trying to use filebeat's multiline support to combine Node.js exceptions into a single message. However, all errors start with a date and there is no unique identifier to create a pattern for this error log. I was wondering if there are any tips I can use to create a pattern that only consolidates error messages to one message. Anything I have tried so far has combined almost any other log message with the same timestamp into one message.

This is sample error log :

count the logs that I mentioned earlier and end the pattern exactly after last line Nov 20 23:45:22 ip-172-31-1-132 web[235438]:  throw new Error('💥 Test Exception! 💥');
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:        ^
Nov 20 23:45:22 ip-172-31-1-132 web[235438]: Error: 💥 Test Exception! 💥
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at getErrorCheck (/var/app/current/js/errorcheck.js:2:9)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at Layer.handle [as handle_request] (/var/app/current/node_modules/express/lib/router/layer.js:95:5)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at next (/var/app/current/node_modules/express/lib/router/route.js:137:13)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at Route.dispatch (/var/app/current/node_modules/express/lib/router/route.js:112:3)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at Layer.handle [as handle_request] (/var/app/current/node_modules/express/lib/router/layer.js:95:5)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at /var/app/current/node_modules/express/lib/router/index.js:281:22
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at Function.process_params (/var/app/current/node_modules/express/lib/router/index.js:335:12)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at next (/var/app/current/node_modules/express/lib/router/index.js:275:10)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at /var/app/current/js/index.js:84:5
Nov 20 23:45:22 ip-172-31-1-132 web[235438]:    at Layer.handle [as handle_request] (/var/app/current/node_modules/express/lib/router/layer.js:95:5)
Nov 20 23:45:22 ip-172-31-1-132 web[235438]: Node.js v18.18.0
Nov 20 23:45:22 ip-172-31-1-132 web[235813]: > XXX@3.8.35 start
Nov 20 23:45:22 ip-172-31-1-132 web[235813]: > node -r esm -r dotenv/config js/index.js

the last thing I tried was defining a unique pattern as follows but my pattern that doesn't combine any messages at all :

filebeat.inputs:
- type: filestream
  id: my-filestream-id
  enabled: true
  paths:
    - /var/log/web.stdout.log
  enabled: true
  parsers:
    - multiline:
        type: pattern
        pattern: '^\d{2}-\w{3}-\d{4}.*throw new Error:'
        negate: false
        match: after
        multiline.flush_pattern: '^> node'
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.