Hi I was wondering if it was possible to get more information about how logstash processes the teardown. I'd imagine it first tearsdown the input, then filters, then output. If there are messages in the filter queue, do these get finished processing through the pipeline (out to output) before the shutdown? Or are those just simply lost?
It tries to flush anything that lives in it's queue, not sure on the rest though.
EDIT: Actually, another question. Is it possible to flush a last message out from my filter into the output?
Or, is it possible for to set logstash such that on a shutdown call, it will just stop reading the inputs, and let all the messages in the queue to run through logstash (filter->output)? Or does that do it by default?
Logstash 1.5 by default stops all inputs and tries to flush all messages currently in the internal queue (20 messages, tops) before shutting down in response to SIGTERM or SIGINT. See the shutdown and run methods in pipeline.rb.
Logstash 1.4.2 only tried a graceful shutdown in response to SIGINT; SIGTERM means an abrupt shutdown.
@kelvinfann Below is a link to github issue describing the details of the shutdown process as it relates to input plugins. This very good explanation of the shutdown sequence shed light on why I was seeing my input plugin 'teardown' method being called twice.
Thanks for the help! @fh8510 I actually ran into the same issue of teardown being called twice as well and was rather confused until I found that git issue as well. Also thanks @magnusbaeck for consistently helping me out.