# How to load balance data coming from various applications to logstash **URL:** \ **Category:** Logstash\ **Created:** [November 29, 2022, 5:17am UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989 "2022-11-29T05:17:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [November 29, 2022, 5:17am UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/1 "2022-11-29T05:17:26Z")
Hi All, I need some suggestions on how/what can be used to balance the data that is coming from 1000 of applications to logstash. Architecture looks like this: We have various sources who's logs needs to be integrated with Elastic search, The incoming logs needs to be distributed across 8 logstash servers. Currently we are using F5 load balancer for distributing the incoming data from sources to logstash servers , but we want to remove or completely replace the F5 load balancer. Please share your ideas. TIA Regards, Anusha K ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [November 30, 2022, 1:02am UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/2 "2022-11-30T01:02:07Z")
The best option would be to use a load balancer, so if you remove the F5 you will probably want to find a replacement for it. ---
**Author:** ![Anusha\_Kusanghi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anusha_kusanghi/32/98109_2.png) [@Anusha\_Kusanghi](https://discuss.elastic.co/u/Anusha_Kusanghi)\ **Post date:** [November 30, 2022, 4:44am UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/3 "2022-11-30T04:44:26Z")
Can we use DNS records that maps to these 8 Logstash shippers, to distribute the incoming logs over TCP protocol. ---
**Author:** ![Sunile\_Manjee](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sunile_manjee/32/111461_2.png) [@Sunile\_Manjee](https://discuss.elastic.co/u/Sunile_Manjee)\ **Post date:** [November 30, 2022, 4:51am UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/4 "2022-11-30T04:51:54Z")
Maybe consider a message broker, Apache Kafka. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 30, 2022, 5:38pm UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/5 "2022-11-30T17:38:29Z")
> [@Anusha\_Kusanghi](#): > > Can we use DNS records that maps to these 8 Logstash shippers, to distribute the incoming logs over TCP protocol. Maybe, maybe not. If a domain name resolves to a list of IP addresses then the DNS server should return them in a different order for each request it serves. But there are multiple levels of caching of the responses, so in practice the order may be the same. It really depends on the software stack you are using to resolve addresses. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [November 30, 2022, 7:23pm UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/6 "2022-11-30T19:23:20Z")
> [@Anusha\_Kusanghi](#): > > Currently we are using F5 load balancer for distributing the incoming data from sources to logstash servers , but we want to remove or completely replace the F5 load balancer. If you remove the F5 you would need something similar to Load Balance the requests, maybe a server with HAProxy, but then to have resilience you would need more than one server and a VIP address using Keepalived. Or you may change the way you ingest your data and send everything to a Kafka Cluster and configure your Logstash servers to read from that Kafka Cluster. I would not recommend using DNS to load balance because of cache and other things, it is not really a load balancer ---
**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\ **Post date:** [November 30, 2022, 7:32pm UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/7 "2022-11-30T19:32:23Z")
I use Haproxy for this reason. sends thousands of requests to five logstash server and it does good job. sends almost equal amount of record to each logstash server. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 28, 2022, 7:32pm UTC](https://discuss.elastic.co/t/how-to-load-balance-data-coming-from-various-applications-to-logstash/319989/8 "2022-12-28T19:32:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.