A Failed Logstash pipeline is causing other pipelines to fail!

Elastic Stack Logstash
1

I have Logstash V 7.3.0 running with 4 pipelines:

pipeline_a.conf
pipeline_b.conf
pipeline_c.conf
pipeline_d.conf

and they're all in the same folder.

When pipeline_a.conf is failing for some reason, which it's not my question, but having it failed is causing the other pipelines (b, c and d) failing and all of them down!

Note: I don't have any logstash.yml or pipelines.yml set in the logstash instance I'm running.

Update:
I checked the documentation and some topics in here, but I couldn't figure it out how to start and what to add to configure one Logstash instance to run multiple pipelines!

2

If you have multiple config files in a single directory and do not use pipelines.yml all files are concatenated into a single pipeline. Unless you use conditionals to control flow data from all inputs will go to all outputs, and this is also why one config failing affects the entire pipeline.

3

@Christian_Dahlqvist thanks for the reply!
What if i put them in a separate folders? would that help instead of having the pipelines.yml file because it's difficult to create it! i couldn't figure it out.

4

No, using the pipelines.yml file is the only way to have independent pipelines within a single Logstash process.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.