How does Filebeat manage problems connecting to Logstash?

Elastic Stack Beats
1

Hello all,

I'd like to know how Filebeat behaves when it can't connect with Logstash.

From the libbeat documentation, in case Logstash is down (or there is a network problem that prevent it reaching Logstash), Filebeat waits for an answer for a specific time defined by the timeout parameter. In this case, it will retry for a maximum number defined by max_retries. In case it doesn't succeed either, libbeat documentation says: "If an event is dropped by the output plugin, each Beat implementation must determine whether to drop the event or try sending it again." What is the default behaviour of Filebeat?

In the previous situation, imagine there are 30 events that should be sent and the first delivery fails. Will it try the whole same process for the remaining 29 events or are they treated as a package and if one fails, the whole package fails?

2

Filebeat guarantees send at least once for every single event. If Logstash is not available internal queues in filebeat will fill up and filebeat stops reading from already opened files until Logstash is available again (or some other logstash instance is available).

1 Like