I have already posted about the startup/shutdown order when using multiple pipelines. Till v6.2.4, it seems that Logstash arranged the startup/shutdown of pipelines in alphabetic order.
I recently upgraded to 6.3.0 and it seems the startup/shutdown is no longer ordered (or maybe done at the same time using threads). So, I decided to switch to the new (beta) "pipeline" output to join two pipelines together. My understanding was that logstash would enforce ordering by which the upstream pipeline shuts down first and starts up last so that no data loss occurs.
My pipeline setup is
0_main (Input and processing) ---> 1_hadoop (Dedicated pipeline to send events to webhdfs)
---> indicates a pipeline input/output glue (0_main has a pipeline output to a virtualaddress "foo" and 1_hadoop defines a pipeline input at virtualaddress "foo").
When I started this setup, 1_hadoop was started first but during shutdown, it was also shutdown first, which led the 0_main to stall at shutdown with errors like
[2018-06-29T16:29:51,085][WARN ][org.logstash.plugins.pipeline.PipelineBus] Attempted to send event to 'foo' but that address was unavailable. Maybe the destination pipeline is down or stopping? Will Retry.
Eventually the timeouts kicked in and data was purged and shutdown was completed.
I have two questions:
- Is there a way to enforce startup shutdown ordering for pipelines so that shutdown stalls can be prevented?
- When using "pipeline" plugin as glue, are there any plans to enforce an implicit ordering (atleast of the glued pipelines) so that the downstream pipeline starts first and terminates last (which to my mind is the logical way connected pipelines should behave)?