# Why mutate will influence different pipelines? **URL:** \ **Category:** Logstash\ **Created:** [March 22, 2023, 6:10am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210 "2023-03-22T06:10:06Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 22, 2023, 6:10am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/1 "2023-03-22T06:10:06Z")
Hi I'd like to process the same input data in different ways to get different results, so I'm trying to do with multiple pipelines. However, I notice that using mutate will influence other pipelines. Does anyone know how to avoid this situation? Thanks in advance! Here is my pipelines.yml ```auto - pipeline.id: upstream config.path: "/usr/share/logstash/pipeline/upstream.conf" - pipeline.id: first-aggregator-downstream queue.type: persisted config.path: "/usr/share/logstash/pipeline/first-aggregator.conf" - pipeline.id: second-aggregator-downstream queue.type: persisted config.path: "/usr/share/logstash/pipeline/second-aggregator.conf" ``` Here is my upstream.conf ```auto input { generator { lines => [ "line1" ] count => 1 } } output { pipeline { send_to => [ firstAggregatorAddress, secondAggregatorAddress ] } } ``` Here is my first-aggregator.conf ```auto input { pipeline { address => firstAggregatorAddress } } filter { mutate { add_field => { "first" => "first" } } } output { stdout { codec => json } } ``` Here is my second-aggregator.conf ```auto input { pipeline { address => secondAggregatorAddress } } filter { mutate { add_field => { "second" => "second" } } } output { stdout { codec => json } } ``` The results are ```auto { "@version" => "1", "host" => { "name" => "hostname" }, "second" => [ [0] "second", [1] "second", [2] "second", [3] "second", [4] "second" ], "message" => "line1", "event" => { "original" => "line1", "sequence" => 0 }, "first" => [ [0] "first", [1] "first", [2] "first", [3] "first", [4] "first" ], "@timestamp" => 2023-03-22T05:47:51.940537527Z } ``` ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 2:42am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/2 "2023-03-31T02:42:59Z")
Can anyone help? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 31, 2023, 2:57am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/3 "2023-03-31T02:57:14Z")
> [@AlanChan](#): > > However, I notice that using mutate will influence other pipelines. Does anyone know how to avoid this situation? Something is not right, when you use `pipelines.yml` the pipelines are independent from each other, so the input, filters and output cannot interfere with other pipelines. How are you running logstash? What do you have in your logstash logs? Please share your logs. Also, it is `path.config`, not `config.path`. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 31, 2023, 3:14am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/4 "2023-03-31T03:14:54Z")
Just replicate your pipeline and it is working fine, the output you shared is not what the pipelines you shared would generate, so it is probably related to how you are running logstash. This is the output I got, and what you should've got since you are using the codec in the stdout output as `json`. ```auto {"event":{"sequence":0,"original":"line1"},"@timestamp":"2023-03-31T03:10:34.298362520Z","@version":"1","message":"line1","host":{"name":"weiss"},"first":"first"} {"event":{"sequence":0,"original":"line1"},"@timestamp":"2023-03-31T03:10:34.298362520Z","@version":"1","message":"line1","host":{"name":"weiss"},"second":"second"} ``` ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 3:59am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/5 "2023-03-31T03:59:27Z")
Thank you for confirming that. I'll try it. ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 4:00am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/6 "2023-03-31T04:00:58Z")
Oh, I see. But that's weird that it doesn't show any error config messages. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [March 31, 2023, 4:15am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/7 "2023-03-31T04:15:28Z")
> [@AlanChan](#): > > Oh, I see. But that's weird that it doesn't show any error config messages. You probably are not using the `pipelines.yml`, but you didn't share how you are running Logstash or any logs. ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 4:53am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/8 "2023-03-31T04:53:35Z")
I'm using k8s to run logstash and mount aforementioned config files to /usr/share/logstash/pipeline. What I can't understand is why there are added field values if the pipelines.yml is not used. ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 6:21am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/9 "2023-03-31T06:21:41Z")
Thank you @leandrojmp . Thanks to you. I finally noticed the message: ```auto {"level":"WARN","loggerName":"logstash.config.source.multilocal","timeMillis":1680243606102,"thread":"LogStash::Runner","logEvent":{"message":"Ignoring the 'pipelines.yml' file because modules or command line options are specified"}} ``` This is the cause of problem. ---
**Author:** ![AlanChan](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AlanChan](https://discuss.elastic.co/u/AlanChan)\ **Post date:** [March 31, 2023, 6:44am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/10 "2023-03-31T06:44:11Z")
Finally I made it work by removing **path.config** from logstash.yml. Sorry for not providing this here. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 28, 2023, 6:44am UTC](https://discuss.elastic.co/t/why-mutate-will-influence-different-pipelines/328210/11 "2023-04-28T06:44:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.