The log message below are two node stack traces. I need the multiline message to be seen in Kibana as a single message.
Below the logs is the logstash configuration that is kind of working but is cutting off the top most line in the stack trace. As you can see I've specified the different log levels that could be sent. The vast majority are INFO: which are being processed correctly.
What do I need to add/modify in logstash.conf to maintain the easy multiline parsing that's working now but also be able to handle the different stack traces below?
Thanks for your help.
Stack trace #1
2018-10-08T07:02:13+00:00 ip-10-xxx—xxx-xxx.ec2.internal purchase-microservice[10946]: (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'reference' of undefined
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at Object.<anonymous> (/payment-providers/obfuscated/messages.js:70:78)
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at step (/payment-providers/obfuscated/messages.js:32:23)
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at Object.next (/payment-providers/obfuscated/messages.js:13:53)
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at fulfilled (/payment-providers/obfuscated/messages.js:4:58)
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at <anonymous>
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at process._tickCallback (internal/process/next_tick.js:188:7)
2018-10-08T07:02:13+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
Stack trace #2
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: error: TypeError: Cannot read property 'userId' of undefined
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at Object.<anonymous> (/payment-providers/obfuscated/routes.js:236:78)
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at step (/payment-providers/obfuscated/routes.js:32:23)
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at Object.next (/payment-providers/obfuscated/routes.js:13:53)
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at fulfilled (/payment-providers/obfuscated/routes.js:4:58)
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at <anonymous>
2018-10-10T22:21:08+00:00 ip-10-xxx-xxx-xxx.ec2.internal purchase-microservice[10946]: at process._tickCallback (internal/process/next_tick.js:188:7)
logstash.conf - relevant to this log file
# Purchase microservice
file {
path => "/var/log/purchase/purchase.log"
type => "purchase-microservice"
tags => ["purchase-microservice"]
codec => multiline {
pattern => "^.*node:|^.*info:|^.*emerg:|^.*alert:|^.*crit:|^.*warning:|^.*notice:|^.*debug:|^.*error:|^.*warn:|^.*verbose:|^.*debug:|^.*silly:"
negate => true
what => next
max_lines => 15
}
}