# Move to pipeline-to-pipeline communiction

**URL:** <https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379>\
**Category:** Logstash\
**Created:** [April 3, 2020, 9:59am UTC](https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379 "2020-04-03T09:59:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [April 3, 2020, 9:59am UTC](https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379/1 "2020-04-03T09:59:07Z")

</div>

Hi,

Today we have a pretty complicated single pipeline comprised mostly of a lot of conditionals to route everything to its right place. One issue with this that we've noticed is if a output goes down, it affects the whole pipeline.

What we want to do is the following:

Send from logstash to two different elasticsearch endpoints, but the same documents.

Today an example output looks like this:

```auto
    else if "syslog" in [tags] and "sql_successful" in [tags] and [cendotServiceName] == "service-audit" and [vd] == "vd01" {
       elasticsearch {
                                    ilm_enabled => true
                                    ilm_rollover_alias => "daily"
                                    ilm_pattern => "000001"
                                    ilm_policy => "SIZE-30_AGE-30_DEL-365"
                                    hosts => ["10.229.1.12:9200", "10.229.1.13:9200"]
                                    user => logstash_internal
                                    password => password
                    }
       elasticsearch {
                                    #manage_template => false
                                    hosts => ["192.168.1.10:9200"]
                                    index => "logs_write"
                                    user => "admin"
                                    ilm_enabled => "false"
                                    password => "password"
                                    ssl => true
                                    ssl_certificate_verification => false
                    }
    }

```

How would we go about changing this configuration to support a second pipeline? We basically want to move the "second" elasticsearch output to a new pipeline. I would imagine something like this?

```auto
    else if "syslog" in [tags] and "sql_successful" in [tags] and [cendotServiceName] == "service-audit" and [vd] == "vd01" {
           elasticsearch {
                                        ilm_enabled => true
                                        ilm_rollover_alias => "daily"
                                        ilm_pattern => "000001"
                                        ilm_policy => "SIZE-30_AGE-30_DEL-365"
                                        hosts => ["10.229.1.12:9200", "10.229.1.13:9200"]
                                        user => logstash_internal
                                        password => password
                        }
           pipeline { send_to => "second_pipeline" }
        }

```

Where "second\_pipeline" just contains the following?

```auto
    output {
       elasticsearch {
                                        #manage_template => false
                                        hosts => ["192.168.1.10:9200"]
                                        index => "logs_write"
                                        user => "admin"
                                        ilm_enabled => "false"
                                        password => "password"
                                        ssl => true
                                        ssl_certificate_verification => false
                        }
    }

```

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [April 7, 2020, 5:48am UTC](https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379/2 "2020-04-07T05:48:09Z")

</div>

Anyone?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 7, 2020, 3:16pm UTC](https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379/3 "2020-04-07T15:16:27Z")

</div>

Your suggestion to use a second pipeline looks reasonable. It is basically the [output isolator](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#output-isolator-pattern) pattern. Did you have a problem when you tested it?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2020, 3:16pm UTC](https://discuss.elastic.co/t/move-to-pipeline-to-pipeline-communiction/226379/4 "2020-05-05T15:16:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.