# ELK Active Active Setup with Failure Recovery

**URL:** <https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621>\
**Category:** Logstash\
**Created:** [January 20, 2023, 6:29pm UTC](https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621 "2023-01-20T18:29:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mostafaelsayed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mostafaelsayed/32/82057_2.png) [@mostafaelsayed](https://discuss.elastic.co/u/mostafaelsayed)\
**Post date:** [January 20, 2023, 6:29pm UTC](https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621/1 "2023-01-20T18:29:36Z")

</div>

Hello All

The use case is that we want to setup Active Active architecture in ELK using two clusters in two different regions. We will index the event to both the local elasticsearch cluster and remote elasticsearch cluster.

To achieve failure-recovery, I wanted to use DLQ (Dead Letter Queue) feature in case Elasticsearch is down so I can store them somewhere else in the local cluster until elasticsearch is back up and reprocess those events and re-index them.

After I read through the docs, it seems this is not possible because elasticsearch has to respond with either 400 or 404 to send the event to the DLQ. Is there any other option to achieve this kind of setup with failure-recovery?

What I thought of is to replace my output stages with custom logic in filter stage to index the event and in case elasticsearch is down, I can index the event somewhere else, but I don't know if any problem would appear from that or any considerations I need to have to achieve a performant indexing as I would using output stage.

Any ideas are appreciated.

Thanks

---

<div class="post-metadata">

**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\
**Post date:** [January 21, 2023, 12:07am UTC](https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621/2 "2023-01-21T00:07:28Z")

</div>

Elastic does this in a single cluster using [Cluster-level shard allocation and routing settings | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#shard-allocation-awareness)

Use the region name for the "rack" value. Put a master in each space and I'd put a voting-only master in yet a third region. Put ingest and Kibana in both regions. Allocate 1 replica for each index, replica's won't be housed in the same "rack".

I did that self hosted where we had campuses in various locations state wide and it's worked for several years.

---

<div class="post-metadata">

**Author:** ![mostafaelsayed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mostafaelsayed/32/82057_2.png) [@mostafaelsayed](https://discuss.elastic.co/u/mostafaelsayed)\
**Post date:** [January 21, 2023, 6:34pm UTC](https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621/3 "2023-01-21T18:34:31Z")

</div>

Thanks @Len_Rugen I am not sure if this will help with our setup but I will check it.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 18, 2023, 6:35pm UTC](https://discuss.elastic.co/t/elk-active-active-setup-with-failure-recovery/323621/4 "2023-02-18T18:35:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.