Hi
I'd like to process the same input data in different ways to get different results, so I'm trying to do with multiple pipelines. However, I notice that using mutate will influence other pipelines. Does anyone know how to avoid this situation?
Thanks in advance!
Here is my pipelines.yml
- pipeline.id: upstream
config.path: "/usr/share/logstash/pipeline/upstream.conf"
- pipeline.id: first-aggregator-downstream
queue.type: persisted
config.path: "/usr/share/logstash/pipeline/first-aggregator.conf"
- pipeline.id: second-aggregator-downstream
queue.type: persisted
config.path: "/usr/share/logstash/pipeline/second-aggregator.conf"
Here is my upstream.conf
input {
generator {
lines => [
"line1"
]
count => 1
}
}
output {
pipeline {
send_to => [
firstAggregatorAddress,
secondAggregatorAddress
]
}
}
Here is my first-aggregator.conf
input {
pipeline {
address => firstAggregatorAddress
}
}
filter {
mutate {
add_field => { "first" => "first" }
}
}
output {
stdout { codec => json }
}
Here is my second-aggregator.conf
input {
pipeline {
address => secondAggregatorAddress
}
}
filter {
mutate {
add_field => { "second" => "second" }
}
}
output {
stdout { codec => json }
}
The results are
{
"@version" => "1",
"host" => {
"name" => "hostname"
},
"second" => [
[0] "second",
[1] "second",
[2] "second",
[3] "second",
[4] "second"
],
"message" => "line1",
"event" => {
"original" => "line1",
"sequence" => 0
},
"first" => [
[0] "first",
[1] "first",
[2] "first",
[3] "first",
[4] "first"
],
"@timestamp" => 2023-03-22T05:47:51.940537527Z
}