# Logstash HA

**URL:** <https://discuss.elastic.co/t/logstash-ha/307946>\
**Category:** Logstash\
**Created:** [June 23, 2022, 5:26am UTC](https://discuss.elastic.co/t/logstash-ha/307946 "2022-06-23T05:26:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![priyankamondalhcl](https://avatars.discourse-cdn.com/v4/letter/p/57b2e6/32.png) [@priyankamondalhcl](https://discuss.elastic.co/u/priyankamondalhcl)\
**Post date:** [June 23, 2022, 5:26am UTC](https://discuss.elastic.co/t/logstash-ha/307946/1 "2022-06-23T05:26:11Z")

</div>

Hi Team,

I am using logstash component for collecting/transforming data and then send to pubsubLite. Now there are cases to collect data using push mechanism (other sources are pushing data to logstash component) as well as pull mechanism (logstash is calling data or event through API/ python script).  
Can you please guide/suggest way to achieve HA in logstash while pulling the data.

---

<div class="post-metadata">

**Author:** ![priyankamondalhcl](https://avatars.discourse-cdn.com/v4/letter/p/57b2e6/32.png) [@priyankamondalhcl](https://discuss.elastic.co/u/priyankamondalhcl)\
**Post date:** [June 27, 2022, 5:04am UTC](https://discuss.elastic.co/t/logstash-ha/307946/2 "2022-06-27T05:04:39Z")

</div>

I still looking for good HA solution, pls update

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 27, 2022, 6:58am UTC](https://discuss.elastic.co/t/logstash-ha/307946/3 "2022-06-27T06:58:54Z")

</div>

There's historic threads on this that still apply - there's nothing native and you will need a load balancer.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 27, 2022, 3:58pm UTC](https://discuss.elastic.co/t/logstash-ha/307946/4 "2022-06-27T15:58:29Z")

</div>

If you are using multiple logstash instances to pull the same data from an API and want to avoid duplicates in Pub/Sub Lite that is really not a logstash question. If the output were elasticsearch then you could set the document\_id to a hash of the message fields using a fingerprint filter, which would cause duplicates to overwrite oneanother, but I do not know if Pub/Sub provides a way to de-duplicate data by using a unique message id or some other method.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2022, 3:59pm UTC](https://discuss.elastic.co/t/logstash-ha/307946/5 "2022-07-25T15:59:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.