# How to configure filebeat for logstash cluster environment?

**URL:** <https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851>\
**Category:** Logstash\
**Created:** [April 7, 2020, 8:20am UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851 "2020-04-07T08:20:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![manish.sapariya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manish.sapariya/32/48133_2.png) [@manish.sapariya](https://discuss.elastic.co/u/manish.sapariya)\
**Post date:** [April 7, 2020, 8:20am UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851/1 "2020-04-07T08:20:05Z")

</div>

I am missing something very basic when I think of how Filebeat will be configured in a clustered logstash setup.

As per the article [https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html](https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html) and this architecture diagram in the article, I think that there is some kind of load balancer in front of the logstash cluster. However, the Filebeat output documentation suggests that there must be an array of all the Logstatsh nodes specified. Using this list of nodes, Filebeat will do the load balancing from the client-side.

Also as per [https://github.com/elastic/logstash/issues/2632](https://github.com/elastic/logstash/issues/2632) GitHub issue, there is no native logstash clustering available yet.

So, my question is, what kind of setup do I need to be able to point my multiple Filebeat to one logstash service endpoint without specifying the logstash nodes in the cluster?

- Is it possible?
- Would having load balancer in front of Logstash cluster be of any help?

Thanks,  
Manish

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [April 7, 2020, 8:28am UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851/2 "2020-04-07T08:28:49Z")

</div>

Hello @manish.sapariya

Filebeat is able to load balance across multiple Logstash instances using the `loadbalance` option (see [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#loadbalance)).

It is possible to use a Load balancer, but it requires sticky TCP sessions and enabling `ttl` to ensure the requests are well balanced across all hosts (see [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#_ttl)).

---

<div class="post-metadata">

**Author:** ![manish.sapariya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manish.sapariya/32/48133_2.png) [@manish.sapariya](https://discuss.elastic.co/u/manish.sapariya)\
**Post date:** [April 7, 2020, 8:45am UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851/3 "2020-04-07T08:45:06Z")

</div>

Hi @Luca_Belluccini,  
Thank you for your response. I am aware of the loadbalance option. Here are some specifics which confuses me.

We plan to run the logstash in the AWS autoscale service. As the instances go down and come up dynamically I am not sure how do we keep the filebeat aware of the changes that have taken place in the logstash instances.

Hope this helps understand my question.  
-Manish

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 7, 2020, 2:52pm UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851/4 "2020-04-07T14:52:31Z")

</div>

> [@manish.sapariya](#):
>
> I think that there is some kind of load balancer in front of the logstash cluster

No, the expectation is that filebeat will load balance across an array of logstash endpoints listed in its configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2020, 2:52pm UTC](https://discuss.elastic.co/t/how-to-configure-filebeat-for-logstash-cluster-environment/226851/5 "2020-05-05T14:52:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.