Merge multiple lines into one with Filebeat

Elastic Stack Beats
1

Hi!

I have a Filebeat instance running along with a Logstash server, to which Filebeat data is forwarded.
Before sending data to Logstash, I want to merge certain lines of the watched log file together.

For example, I have this data:
sudo[3]: ...session opened... sudo[3]: ...session closed... sshd[19]: authentication failure... sshd[19]: authentication success...

The two sudo lines are two distinct log messages, but they are still related, because of the number between the square brackets for example. Same for sshd. Is there a way to merge the sudo lines (or sshd lines) so that only one message would be forwarded to Logstash, instead of one for each line? Or, more generally, merge lines that have a value in common?

I've looked into the multiline option, but it doesn't seem to be exactly what I'm looking for, as it seems more adapted to unique log entries that take multiple lines. This is not exactly my case here.

Can somebody help me please?
Thanks in advance
Théo

2

You can't do this with beats, you need to send to LS and use https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

3

Yeah, I actually have seen this Logstash plugin, but for some reason it cannot be used with my setup, so I wanted to see if there was a way to do it with Filebeat nonetheless. I'll just try and find another way. Thank you for your help.

4

This topic was automatically closed after 21 days. New replies are no longer allowed.