# Multiple Elasticsearch Output Pipeline

**URL:** <https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736>\
**Category:** Logstash\
**Created:** [March 23, 2020, 8:57pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736 "2020-03-23T20:57:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marc\_Wolff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc_wolff/32/45055_2.png) [@Marc\_Wolff](https://discuss.elastic.co/u/Marc_Wolff)\
**Post date:** [March 23, 2020, 8:57pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736/1 "2020-03-23T20:57:21Z")

</div>

Hi folks,  
we have a pipeline with beats input and multiple ES Hosts outputs.  
But if one of the ES Hosts unreachable or the index is full, all other Outputs also doesn´t get data.  
Is that normal, or a misconfiguration from our side?

In the log we get only permanent warnings that one of the ES Hosts is not reachable.

Our Setup:  
Kubernetes Cluster  
Elastic Helm Chart  
Beats/Logstash/ES Version 7.6.1

Config:

```auto
     pipelines.yml: |
        - pipeline.id: beats-server
          config.string: |
            input { beats { port => 5044 } }
            output {
              if [kubernetes][namespace] == "bvsecure-trusted" {
                 pipeline { send_to => bvsecure_trusted }
              } else if [kubernetes][namespace] =~ "payments" {
                 pipeline { send_to => payments }
              } else {
                 pipeline { send_to => fallback }
              }
            }
        - pipeline.id: bvsecure-trusted-processing
          config.string: |
            input { pipeline { address => bvsecure_trusted } }
            output {
              elasticsearch {
                  hosts => ["estrusted:9200"]
                  password => "XXX"
                  user => "XXX"
                  manage_template => false
                  index => "%{[kubernetes][container][name]}-%{+YYYY.MM.dd}"
                  ssl => true
                  cacert => "/etc/logstash/config/certs/elastic-ca.pem"
              }
            }
        - pipeline.id: payments-processing
          config.string: |
            input { pipeline { address => payments } }
            output {
              elasticsearch {
                  hosts => ["espayments:9200"]
                  password => "XXX"
                  user => "XXX"
                  manage_template => false
                  index => "%{[kubernetes][container][name]}-%{+YYYY.MM.dd}"
                  ssl => true
                  cacert => "/etc/logstash/config/certs/elastic-ca.pem"
              }
            }
        - pipeline.id: fallback-processing
          config.string: |
            input { pipeline { address => fallback } }
            output {
              elasticsearch {
                  hosts => ["esdelivery:9200"]
                  password => "XXX"
                  user => "XXX"
                  manage_template => false
                  index => "fallback-%{+YYYY.MM.dd}"
                  ssl => true
                  cacert => "/etc/logstash/config/certs/elastic-ca.pem"
              }
            }

```

thanks  
marc

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 23, 2020, 10:30pm UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736/2 "2020-03-23T22:30:05Z")

</div>

> [@Marc\_Wolff](#):
>
> Is that normal

Yes. You could use pipeline to pipeline comms with an [output isolator pattern](https://www.elastic.co/guide/en/logstash/current/pipeline-to-pipeline.html#output-isolator-pattern) to resolve it.

---

<div class="post-metadata">

**Author:** ![Marc\_Wolff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc_wolff/32/45055_2.png) [@Marc\_Wolff](https://discuss.elastic.co/u/Marc_Wolff)\
**Post date:** [March 24, 2020, 8:04am UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736/3 "2020-03-24T08:04:12Z")

</div>

Hi Badger,  
thank you very much  
Marc

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2020, 8:04am UTC](https://discuss.elastic.co/t/multiple-elasticsearch-output-pipeline/224736/4 "2020-04-21T08:04:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.