Behaviour with unavailable Gelf output

Elastic Stack Logstash
1

Hi,
how does Logstash behave if a Gelf output is configured but the destination host is temporarily unavailable?
Are messages lost, or are they kept in a temporary place until the target Gelf host is available again?

I am using logstash 1.3.3 on RHEL 6 (and I cannot switch to a newer version).

Thank you.

2

If an output is down Logstash halts the whole pipeline. Whether this causes you to lose messages depends on your inputs. Any UDP messages will be dropped, but file inputs are okay, as are TCP inputs as long as the sending party treats interruptions gracefully.

3

Hi, well the only input is syslogng via tcp, so if I understand correctly messages should not be lost, right?
As far as I know syslog-ng will hold any pending messages.

So... logstash checks the availability of the output before accepting any messages from the input?
If this is not the case, what happens if a messages has been received but then the output is not available?

Thank you very much.

4

Hi, well the only input is syslogng via tcp, so if I understand correctly messages should not be lost, right?
As far as I know syslog-ng will hold any pending messages.

I'm not familiar with syslog-ng's behavior in those cases. It obviously also depends on the duration of the Logstash outage.

So... logstash checks the availability of the output before accepting any messages from the input?
If this is not the case, what happens if a messages has been received but then the output is not available?

Logstash has an internal queue of 20 items. If the queue is full, inputs that want to submit message block. For file inputs this simply means that they stop reading from the input file, and for TCP and UDP inputs it eventually means that they'll stop accepting connections/messages.

Items already in the queue will eventually be flushed, unless Logstash is shut down. If that happens the queued messages will be lost.

5

Thank you, this point is very useful to know.
What I was trying to understand is whether a service restart of the full logstash+graylog stack can cause the loss of incoming messages.
So I now see it's better if I first stop logstash, then stop graylog etc., then restart graylog etc. and at last restart logstash.