Hi. I'd like to talk about one case, that had been today and maybe get and advice in pipeline handling.
Today I added new ES output to my logstash config. I define outputs in separate pipelines to avoid problem when one dead output hangs all another outputs. I use forked-path pattern for this.
I spread configs with new pipeline across my logstash servers and made logstash service reload configuration.
After several hours I found out that logstash queue at one server is growing. I noticed that server was not providing monitoring data via x-pack monitoring. Server log was full of messages about dead ES instance.
The problem was in firewall misconfiguration and I fixed it.
But I have a question now, how did it happen that one output in separate pipeline hung monitoring pipeline and may be other pipelines too?