# Duplicate and forward logs to a new instance of Elasticsearch

**URL:** https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819
**Category:** Logstash
**Created:** [May 26, 2019, 11:39pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819 "2019-05-26T23:39:56Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Camilo\_Diaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_diaz/32/46512_2.png) [@Camilo\_Diaz](https://discuss.elastic.co/u/Camilo_Diaz)
#### Post date: [May 26, 2019, 11:39pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/1 "2019-05-26T23:39:57Z")

</div>

Hi,  
I am fairly new to ELK and I am seeking some advice. I have inherited an ELK stack which is processing logs from multiple sources and I am looking into forwarding some of those logs to another instance of Elasticsearch.

Our current set up is (Cent-OS 7):  
Logstash -\> Elasticsearch (5.6) -\> Kibana

What I want to achieve is to forward some of the logs to a new instance of Elasticsearch and also keep the logs in my current stack. What would be the best way to achieve this?

I was told in the elasticsearch forum that I would have to "duplicate the event to a second output". Any advice is greatly appreciated. Thanks.

---

<div class="post-metadata">

### Author: ![AquaX](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aquax/32/92006_2.png) [@AquaX](https://discuss.elastic.co/u/AquaX)
#### Post date: [May 27, 2019, 8:42pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/2 "2019-05-27T20:42:21Z")

</div>

Add another elasticsearch{} entry to the output{} section of the logstash configuration. Just change the IP/Hostname 🙂

---

<div class="post-metadata">

### Author: ![Camilo\_Diaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_diaz/32/46512_2.png) [@Camilo\_Diaz](https://discuss.elastic.co/u/Camilo_Diaz)
#### Post date: [May 28, 2019, 2:33am UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/3 "2019-05-28T02:33:40Z")

</div>

Thanks AquaX,  
I was able to forward all the output to my 2nd cluster but I believe what I need is a bit different. I'll try to explain:

I want to keep processing all my logs from different sources with logstash and then send them to the localhost instance of elasticsearch. (This is my current setting which I want to keep untouched).

I want to now processs the sames logs with logstash but "extract" and "modify" the same logs, and then send them to my 2nd instance of elasticsearch.

I see this like having another pipeline processing the same logs but using a different output. Is this possible/reasonable?

Thanks in advance!  
Camilo

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 28, 2019, 12:56pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/4 "2019-05-28T12:56:00Z")

</div>

> [@Camilo\_Diaz](#):
>
> I see this like having another pipeline processing the same logs but using a different output. Is this possible/reasonable?

Yes, you could use [pipeline to pipeline](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html) communication to unconditionally send events to two pipelines that process them in different ways.

---

<div class="post-metadata">

### Author: ![Camilo\_Diaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_diaz/32/46512_2.png) [@Camilo\_Diaz](https://discuss.elastic.co/u/Camilo_Diaz)
#### Post date: [June 5, 2019, 1:43am UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/5 "2019-06-05T01:43:12Z")

</div>

I am running logstash 5.6.6 so I guess pipeline to pipeline won't work for me.

I have different config files for every input and one output file pointing to localhost.

> output {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> }

In my output file I have tried to filter by type and then send those types to my 2nd instance of elasticsearch.  
I also configured a completely new input with an output to my 2nd instance of elasticsearch.  
In both scenarios my filter did not work and all the information (from all my inputs) was sent to my 2nd instance of elasticsearch as well.

I am a bit lost on how to move forward with this.

edit: typo

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [June 5, 2019, 12:49pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/6 "2019-06-05T12:49:14Z")

</div>

> [@Camilo\_Diaz](#):
>
> I am running logstash 5.6.6 so I guess pipeline to pipeline won't work for me.

You could try using lumberjack outputs and a beats inputs to the same effect.

---

<div class="post-metadata">

### Author: ![Camilo\_Diaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/camilo_diaz/32/46512_2.png) [@Camilo\_Diaz](https://discuss.elastic.co/u/Camilo_Diaz)
#### Post date: [June 5, 2019, 11:10pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/7 "2019-06-05T23:10:42Z")

</div>

Will give that a try and let u know how it goes

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 3, 2019, 11:10pm UTC](https://discuss.elastic.co/t/duplicate-and-forward-logs-to-a-new-instance-of-elasticsearch/182819/8 "2019-07-03T23:10:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
