Multiline logs with a unique identifier

Elastic Stack Beats
1

Hi All,

I am reading messages from a syslog file in the following format:

<161>notify-1089122.1.alert: abcd
<161>notify-1089122.2.alert: xyz
<161>notify-1089133.1.alert: abc
<161>notify-1089122.3.alert: qwer

These are actually, multi line messages that are broken down by the syslog that is sending these into parts. They all have a unique identifier that tells, that they are part of the same event. How do I convert them to a single line?

The usual patterns do not work in this scenario. I need FileBeat to compare the lines using the unique identifier.

Regards,
Nachiket

2

@NerdSec I think you want the following output?

<161>notify-1089122.1.alert: abcd xyz qwer
<161>notify-1089133.1.alert: abc
3

Thank you for the reply pier. Yes, that is exactly the intended output.

4

@NerdSec currently multiline doesn't support aggregating out of order events which is what you need, you might be able to do it with Logstash and the [Logstash aggregate filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

5

Thanks for the confirmation pier. I ended up receiving logs over http and parsing the corresponding json, using the http input in logstash.