Intermittent log loss observed in a UDP-based Logstash architecture behind a load balancer, with ongoing investigation into possible packet drops, idle-timeout behavior, and UDP input tuning recommendations.

Elastic Stack Logstash
1

Hi everyone,

We are currently troubleshooting intermittent log loss in a Logstash setup using the UDP input plugin and wanted to check if anyone in the community has faced similar behavior.

Current setup (genericized):
Source Systems → Load Balancer → Multiple Logstash Nodes (UDP input) → Downstream destinations

Observations:

  • Logs are generated successfully at the source side

  • Some logs intermittently do not appear downstream

  • No explicit errors are visible in Logstash logs

  • Behavior seems more noticeable after periods of inactivity / low traffic

  • Traffic is load balanced across multiple Logstash nodes

We understand UDP itself is connectionless and does not guarantee delivery, but we wanted to understand specifically from the Logstash/community perspective:

  1. Has anyone observed UDP “timeout-like” behavior where logs stop appearing after idle periods?

  2. Are there any known limitations or caveats with the Logstash UDP input plugin under intermittent traffic?

  3. Has anyone implemented any “keepalive”-style approach for long-running low-frequency UDP sources?

  4. Are there best practices when using UDP behind a load balancer with multiple Logstash nodes?

Additional troubleshooting already performed:

  • Comparing source-generated logs vs received logs

  • Forwarding logs in parallel to multiple destinations for correlation

Any recommendations, similar experiences, or tuning suggestions would be greatly appreciated.

Thanks in advance!

2

Welcome to the community @Usha_Nookala

Do you lose any data if there is only one logstash node defined in your load balancer? Or put another way, do you only lose data if you have multiple logstash nodes defined? That would be first thing I'd try to establish.

Indeed, and unfortunately there is no LB/logstash configuration that will turn UDP into a guaranteed delivery protocol.

Might be useful to share the LB config too, if possible.