Pipeline virtual address from Filbeat event field

Is it possible to get the pipeline-to-pipeline virtual address by accessing event fields? Sort of like this:

Filebeat config

filebeat.inputs:
- type: log
  enabled: true
  fields: { instance_id: "first" }
  paths:
    - /home/user/logs/first.log

output.logstash:
  hosts: ["myhost:5044"]

Logstash config

- pipeline.id: gateway
  config.string: |
    input { beats { port => 5044 } }
    output { pipeline { send_to => "%{[fields][instance_id]}" } }

- pipeline.id: first
  config.string: |
    input { pipeline { address => first } }
    filter {
    # do something 
    }
    output { file { path => "/home/syslogdb/first.log" } }

I tried this with Logstash 7.6 but didn't manage to get it working. Is my syntax off or is this not possible? Thanks!

It is not possible. All of the pipelines are created and inter-connected before any events are processed.

1 Like