# High Availibility for Logstash Input Processing

**URL:** <https://discuss.elastic.co/t/high-availibility-for-logstash-input-processing/123653>\
**Category:** Logstash\
**Created:** [March 12, 2018, 11:51pm UTC](https://discuss.elastic.co/t/high-availibility-for-logstash-input-processing/123653 "2018-03-12T23:51:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ninjada](https://avatars.discourse-cdn.com/v4/letter/n/5daacb/32.png) [@ninjada](https://discuss.elastic.co/u/ninjada)\
**Post date:** [March 12, 2018, 11:51pm UTC](https://discuss.elastic.co/t/high-availibility-for-logstash-input-processing/123653/1 "2018-03-12T23:51:14Z")

</div>

hello, i'm currently using a logstash server to process logs in an AWS s3 bucket via the s3 input, and push them to an AWS elasticsearch service cluster.

however, i need it to have production level redundancy ie. high availability on processing these s3 logs, with an instance ready to failover when required and pickup where it left off with no duplicate/overlap, which is the main part i'm not fully across..

i've setup logstash instances with failover previously behind a load balancer with a port check, but thats with multiple servers with beats pushing to it... i'm not quite sure how to approach this situation where the input processing itself needs the redundancy.. any advice?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 13, 2018, 6:55am UTC](https://discuss.elastic.co/t/high-availibility-for-logstash-input-processing/123653/2 "2018-03-13T06:55:21Z")

</div>

Logstash doesn't help you much here. You can't have more than one s3 input reading from the same bucket unless you filter duplicates later on. That's probably the easiest option though.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 10, 2018, 6:56am UTC](https://discuss.elastic.co/t/high-availibility-for-logstash-input-processing/123653/3 "2018-04-10T06:56:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.