# Pipeline To Pipeline Communication inside Rancher

**URL:** <https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743>\
**Category:** Logstash\
**Created:** [October 20, 2020, 10:03pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743 "2020-10-20T22:03:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohsin106](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohsin106/32/65203_2.png) [@mohsin106](https://discuss.elastic.co/u/mohsin106)\
**Post date:** [October 20, 2020, 10:03pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743/1 "2020-10-20T22:03:12Z")

</div>

Hi, I need some guidance on setting up pipeline-to-pipeline communication inside of Rancher. What config files do I need at a minimum? I'm confused as to whether or not I need both logstash.yml and pipelines.yml configured or just one of them. I'm guessing I need both. Logstash.yml file should contain my xpack authentication parameters and my pipelines.yml should contain the multiple pipeline information each with its own set of input, filter, and output plugins?

---

<div class="post-metadata">

**Author:** ![mohsin106](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohsin106/32/65203_2.png) [@mohsin106](https://discuss.elastic.co/u/mohsin106)\
**Post date:** [October 22, 2020, 3:14pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743/2 "2020-10-22T15:14:29Z")

</div>

I believe I figured it out. Logstash.yml is needed, and you can configure the path.queue setting and point it to the volume inside of the Logstash container that you configured as a persistent volume in Rancher. In my case I attached my persistent volume to the /usr/share/logstash/data directory. So I used the same directory to store queue data which you need when you want to implement pipeline-to-pipeline communication using the [forked pattern](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#forked-path-pattern).

> path.queue: "/usr/share/logstash/data"

Then you also need pipelines.yml which will contain something similar to the below:

```
- pipeline.id: pipeline1
   queue.type: persisted
   config.string: |
     input { your input settings go here }
     filter { your filter settings go here }
     output { pipeline { send_to => "my-pl-1" } } ********

```

In order to pass the event from the first pipeline " **pipeline1**" to a second pipeline " **pipeline2**" you need to point the output {} filter of the first pipeline to a "virtual address" (my-pl-1 in this ex.) that the second pipeline will use as it's input:

```
 - pipeline.id: pipeline2
   queue.type: persisted
   config.string: |
     input { pipeline { address => "my-pl-1" } } ********
     filter { your filter settings go here }
     output { your output settings go here }

```

I configured logstash.yml and pipelines.yml in my config-map file and then mounted that config-map to the /usr/share/logstash/config directory inside of the Logstash container.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 19, 2020, 3:14pm UTC](https://discuss.elastic.co/t/pipeline-to-pipeline-communication-inside-rancher/252743/3 "2020-11-19T15:14:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.