HA and Distributed Logstash with multiline codec

Elastic Stack Logstash
1

I am using syslog input plugin with multiline code to ingest messages from routers and Java applications (multiline mainly for Java exceptions).

I need to scale Logstash to be high available and distributed. I can build something like HAProxy between log sources and Logstash nodes to provide load balancing to all Logstash nodes.

What is not clear to me, how it will handle multiline messages like Java exceptions when it will be load balanced to different Logstash nodes?

For example one part of multiline would be sent to Logstash Node 1 and second part of multiline message would be sent to Logstash Node 2. How it will be handled?

2

What is not clear to me, how it will handle multiline messages like Java exceptions when it will be load balanced to different Logstash nodes?

It won't. Relying on a multiline codec for a syslog input sounds like a bad idea in general. Even with a single Logstash instance, is there any guarantee that messages from different hosts won't interleave?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.