# Logstash: Output fails

**URL:** <https://discuss.elastic.co/t/logstash-output-fails/143818>\
**Category:** Logstash\
**Created:** [August 10, 2018, 7:45am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818 "2018-08-10T07:45:15Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 10, 2018, 7:45am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/1 "2018-08-10T07:45:15Z")

</div>

I have two output, kafka and elasticsearch.

I want to have a fail safe with my setup. If one of them stops, the other one can still index a data.

I tried to stop my elasticsearch while the kafka is still running and I found out that the logstash cannot write also in kafka.

Is there any workaround for this?

TIA 😃

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 10, 2018, 8:58am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/2 "2018-08-10T08:58:42Z")

</div>

What's the desired behavior if e.g. ES stops accepting documents? Drop the events on the floor? Queue them somewhere?

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 10, 2018, 8:59am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/3 "2018-08-10T08:59:34Z")

</div>

If E.S. stops, logstash should continue to index the data to kafka.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 10, 2018, 9:19am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/4 "2018-08-10T09:19:27Z")

</div>

And ES will lose the events that were processed while it was down?

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 10, 2018, 10:23am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/5 "2018-08-10T10:23:16Z")

</div>

Yes. Events will not get into ES.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 10, 2018, 11:07am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/6 "2018-08-10T11:07:41Z")

</div>

That's unusual. I don't think the elasticsearch output plugin is capable of dropping events when ES is down so it takes a more complicated setup. I'm actually not sure how that would be done. It's a lot easier to construct something where the elasticsearch output would pick up the events it has missed; just have two Kafka consumers in different Logstash pipelines. They'll consume from Kafka independently and won't affect each other.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [August 10, 2018, 11:10am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/7 "2018-08-10T11:10:09Z")

</div>

I just want to test my environment. Since i have two outputs(es and kafka), i want to know if what will happen if one of the two fails.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 10, 2018, 11:26am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/8 "2018-08-10T11:26:41Z")

</div>

But you already know the answer to that question. One output with a problem will halt all other outputs in the same pipeline.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 7, 2018, 11:26am UTC](https://discuss.elastic.co/t/logstash-output-fails/143818/9 "2018-09-07T11:26:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
