How does logstash handle multiline logs in a load balancing configuration

Elastic Stack Logstash
1

Hi all,

We collect our infrastructure logs with filebeat on elastic cloud.
We'd like to add a logstash cluster (multiple logstash instances load-balanced) between our filebeat agents and our elastic cloud cluster. The goal here is to leave all the parsing and data transform to logstash. However, I think we might have a problem when it comes to handle multiline logs (like exceptions, error, etc). We'd like it to be in only one document and I think the load balancing might break this.

I know we can handle it from filebeat directly but I was wondering if it was possible to let Logstash do it in this particular configuration.

Thanks !

2

If you have multiline logs you need to configure it in the collector, in this case, you need to configure it in Filebeat, not on Logstash.

This is in the documentation.

If you are sending multiline events to Logstash, use the options described here to handle multiline events before sending the event data to Logstash. Trying to implement multiline event handling in Logstash (for example, by using the Logstash multiline codec) may result in the mixing of streams and corrupted data.

3

It was pretty clear in the doc actually, I must have missed it ! Thank you !

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.