Logstash verify output availability


(João Paulo Karol Nunes) #1

Hi all,
I wonder if there is a way to verify if a tool (elasticSearch etc) defined in the output filter is available before sending messages.
The reason is because logstash is hanging when an output is not available and cannot follow the other output flows;
Does anyone have any idea?


(Magnus Bäck) #2

I wonder if there is a way to verify if a tool (elasticSearch etc) defined in the output filter is available before sending messages.

Sending messages from where?

The reason is because logstash is hanging when an output is not available and cannot follow the other output flows;

Yes, this is by design. Logstash requires all outputs to accept events, Otherwise the pipeline will pause.

It might be easier to help if you explain the bigger picture of what you're trying to do.


(João Paulo Karol Nunes) #3

Hi Magnus,
At some deployment environments, the output hosts defined will NOT be available, but I sill want logstash to send data to the other output hosts defined.
My logstash output filter should be smart enough to ignore sending message if a server is down.


(Magnus Bäck) #4

Logstash currently doesn't support this scenario. You'd have to temporarily disable those outputs.


(João Paulo Karol Nunes) #5

I was thinking I could create a custom output plugin and change the code to check the host is up.
What do you think?


(Magnus Bäck) #6

I don't understand what you mean. How would it help to have a custom output plugin that checks whether the server is up?


(João Paulo Karol Nunes) #7

I am currently using a syslog output plugin.
Checking its ruby code (syslog.rb) I can see the connection code.
I was thinking I could change the code to ignore when the destination is not available and continues the output pipeline event flow.


(Magnus Bäck) #8

Oh, you mean modify an existing plugin to return success even though it failed? Sure, that'd work.


(João Paulo Karol Nunes) #9

Yes that is what I mean. Thanks, let me see if I can get it working.


(Divya Prakash) #10

Hello, we are also facing same issue with Logstash 5.4.0 as well..Wondering if there is any fix available for this yet?