Logstash conf file, difference between multiple input/output vs consolidated input/output

[Baygon (Vincent T)]

Hi,

I'm having about 25 pipelines to configure.

I used to have all my jdbc sources inside one input block, then in the output I would have a if condition on the 'tags' value.
That would mean on input with 25 jdbbc sections and one output with 25 if conditions

What would be the difference in setting all the jdbc as a separate input/output, ie 25 input blocks and 25 output blocks with 1 if condition each ?

Would that impact performances, heap usage, memory release, etc... ?

[Badger]

If you put them all in one input block you do not have 25 pipelines, you have one. If you put them in 25 input blocks and run them in the same pipeline that is equivalent. If you use pipelines.yml and run them in 25 pipelines then the performance will be different. Probably more memory use, but you would have to try it and measure things.

More importantly, if you have 25 separate flows from input to output then having 25 pipelines with a single input and output is much easier to understand.