Distributed Pipeline - Can the send_to address be resolved from a variable?

Elastic Stack Logstash
1

I have a working setup, where events come in from a single filebeat node, hit the log stash main pipeline which sends the event to one of 20 different pipelines.

To do this there is an ugly if statement that does 20 string comparisons and sends the event off..

I'd rather replace that with something like:

pipeline { send_to => "%pipeline_name" } }

When I try this log stash logs "$pipeline_name" is not available.. so it appears its not translating the variable.. is this possible?

2

Pipelines are configured and connected during initialization, so they cannot use sprintf references to events like %{pipeline_name}. Are you trying to do that or to use an environment variable? (I doubt that works either, to be honest.)

3

Thanks, that makes a lot of sense. I was just trying to use logstash variables.. my Filebeat tags the pipeline name on each event so it can get sent to where it needs to go.