So I am trying to use the Logstash-tcp-output plugin to send data from one logstash service to another using the logstash-tcp-input plugin. Everything works just fine when the receiving logstash tcp input is up and running, however when it is not (node goes down or logstash crashes or something unexpected happens) the sending logstash-tcp-output has to recreate the tcp connection and send another SYN. However, this is never ACKed as the receiving tcp input is down. Normally this would be fine.. and I would expect to lose data here until the receiving node recovers... but it seems that when it is in this state, the sending logstash blocks the entire output pipeline.
In my situation, this is a problem, as I would like the sending logstash to still correctly receive inputs and push to its other outputs (ie elasticsearch)... but that doesn't seem to be possible.
Is there a way to not make the tcp output plugin block the entire pipeline when a connection cannot be established. Or is there another output plugin I should be using for my scenario. Would the UDP plugin do what I desire?