# Take events from one pipeline and forward to another pipeline : Pipeline to Pipeline Communication

**URL:** https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903
**Category:** Logstash
**Created:** [November 15, 2018, 4:10pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903 "2018-11-15T16:10:50Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [November 15, 2018, 4:10pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/1 "2018-11-15T16:10:50Z")

</div>

Hi All

I have a scenario, wherein i need to read events from one data source and transfer the output of this pipeline to other pipeline as a sprintf like %{abc}, please let me know how to achieve this

Here is my example conf file

> **pipeline1.cfg**  
> input{  
> file {  
> path =\> ".../flatfile.csv"  
> #sincedb\_path =\> ".../sincedb.txt"  
> type =\> "csv"  
> start\_position =\> "beginning"  
> }  
> }  
> filter {  
> csv {  
> columns =\> ["abc","def","ghi"]  
> separator =\> ","  
> }
> 
> }  
> output { elasticsearch { ... } }

This is second pipeline

> # pipeline2.cfg
> 
> input {  
> http\_poller{  
> urls =\> {  
> http\_request =\> {  
> method =\> get  
> url =\> "[https://abcdefghjo](https://abcdefghjo).........."  
> headers =\> {  
> "authorization" =\> " **%{abc}**"  
> "Content-Type" =\> "application/xml; charset=UTF-8"  
> }  
> }  
> }
> 
> ```
> }
> 
> ```
> 
> }  
> filter { grok { ... } }  
> output { elasticsearch { ... } }

In the second pipeline, i am applying the authorization value %{abc}, which is coming from pipeline1, how to achieve this?

Thanks!

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 15, 2018, 4:14pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/2 "2018-11-15T16:14:10Z")

</div>

You can not feed incoming events into an input plugin (they would come through the pipeline input and go straight to the filters), so I don't think what you are trying to do is possible. For this to work you would probably need some kind of http\_poller filter, but as far as I know no such filter exists.

---

<div class="post-metadata">

### Author: ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [November 15, 2018, 4:18pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/3 "2018-11-15T16:18:07Z")

</div>

Thanks for your reply

This is common scenario, i have lot of uses cases like this, wherein i need to talk with 2 pipelines. I thought this is possible with new 6.x version.

This is i am doing because my Authorization token expires every time. Do you have any solution wherein i can keep my authentication ALIVE?

Thanks

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 15, 2018, 4:19pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/4 "2018-11-15T16:19:14Z")

</div>

You can send data between pipelines, but not trigger input plugins based on the data.

---

<div class="post-metadata">

### Author: ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [November 15, 2018, 4:20pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/5 "2018-11-15T16:20:52Z")

</div>

okay not sure what to do, looks like this is not right tool to achieve this.  
Anyways .Thanks for your response

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [November 15, 2018, 4:22pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/6 "2018-11-15T16:22:43Z")

</div>

One option might be to create a custom plugin, possibly based on the http\_poller plugin, which seems close to fitting your requirements.

---

<div class="post-metadata">

### Author: ![rkhapre](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rkhapre/32/48333_2.png) [@rkhapre](https://discuss.elastic.co/u/rkhapre)
#### Post date: [November 15, 2018, 4:36pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/7 "2018-11-15T16:36:19Z")

</div>

Yes, you are correct. I started working in same direction. But i am not expert of plugin development.  
I am trying something with ruby filter script wherein i can execute http

Thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 13, 2018, 4:36pm UTC](https://discuss.elastic.co/t/take-events-from-one-pipeline-and-forward-to-another-pipeline-pipeline-to-pipeline-communication/156903/8 "2018-12-13T16:36:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
