Retry: Pipeline to Pipeline Output Variable Expansion

#1

Hello,

I have the same question as described in this post.

Since there hasnt been any replies I thought i would just open the same topic again.
I am currently using Logstash-7.3.1 with this configuration for all incoming data:

output {
if [type] {
pipeline { send_to => "%{[type]}" }
}
else {
pipeline { send_to => "default" }
}
}

This is supposed to work like The distributor pattern, but without me having to add a "else if " statement for each new pipeline. My goal is that every event with a field called "type", should send their data to the matching pipeline (virtual address = type). Also if there is no type field defined it should go to the default pipeline.

However i am getting the following error:
Attempted to send event to '%{[type]}' but that address was unavailable. Maybe the destination pipeline is down or stopping? Will Retry.

I appreciate any kind of feedback on this.

Thanks,
Lukas L.

#2

You cannot use a sprintf reference in the name of a pipeline. pipeline connections are created at startup, not on-demand as events arrive.

1 Like
#3

Ok thanks.

Do you have any suggestions on how i could make the pipeline to pipeline communication more dynamic other than using what i have tried?

If not I will more or less have to stick to the multiple "if-statement method" right?

#4

I think your only option is a collection of if-else