# Possible to loadbalance Logstash via DNS?

**URL:** <https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133>\
**Category:** Logstash\
**Created:** [July 2, 2018, 7:17am UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133 "2018-07-02T07:17:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![rcoundon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcoundon/32/30010_2.png) [@rcoundon](https://discuss.elastic.co/u/rcoundon)\
**Post date:** [July 2, 2018, 7:17am UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133/1 "2018-07-02T07:17:00Z")

</div>

I have a number of remote instances of Logstash that send logs over https to an Elasticsearch cluster.  
I'm looking to find a way to loadbalance the requests across the nodes in the cluster ideally without having to specify separate subdomains for each and list them.  
I.e.  
I'd have [https://elk.mycluster.com](https://elk.mycluster.com) with 2 DNS A records that point to 123.123.123.1 and 123.123.123.2 and the DNS provider will round-robin the requests across each.

That's rather than setting up a DNS of:  
[https://elk1.mycluster.com](https://elk1.mycluster.com) for 123.123.123.1  
[https://elk2.mycluster.com](https://elk2.mycluster.com) for 123.123.123.2  
and setting the logstash output to use multiple targets like:

```auto
  elasticsearch {
    host => ["https://elk1.mycluster.com", "https://elk2.mycluster.com"]
    ...
  }
}

```

What's the right way to go here?

---

<div class="post-metadata">

**Author:** ![paz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paz/32/28003_2.png) [@paz](https://discuss.elastic.co/u/paz)\
**Post date:** [July 2, 2018, 10:54am UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133/2 "2018-07-02T10:54:32Z")

</div>

Not sure it could work in your setup, but you could provide Logstash with a single host and use the [sniffing](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-sniffing) parameter to have it auto-populate the list with all the hosts participating in that cluster.

Then Logstash would take care of the load-balancing itself.

---

<div class="post-metadata">

**Author:** ![rcoundon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcoundon/32/30010_2.png) [@rcoundon](https://discuss.elastic.co/u/rcoundon)\
**Post date:** [July 2, 2018, 12:35pm UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133/3 "2018-07-02T12:35:27Z")

</div>

Thanks Paris, are you able to say more about why it won't work? Just interested to understand before I change the architecture.

Thanks

---

<div class="post-metadata">

**Author:** ![paz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paz/32/28003_2.png) [@paz](https://discuss.elastic.co/u/paz)\
**Post date:** [July 2, 2018, 1:16pm UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133/4 "2018-07-02T13:16:42Z")

</div>

Off the top of my head, the 2 major concerns would be:

1. Having dedicated master-eligible nodes in your cluster.  
Those nodes cannot handle bulk indexing requests but they are still included in the _hosts_ list produced by the sniffing parameter.  
However, you need to explicitly define some nodes as master-eligible, so if you haven't, your nodes are most likely both master + data nodes and you should be fine.

2. I'm not sure whether the sniffing option will return the IP of each node or it's hostname. If the latter, you should still need DNS resolution, so it might not be much easier it is than your initial idea.

Edit: By "initial idea" I mean having separate DNS records for each node and passing them onto the hosts lists. In any case you don't need any for of load balancing in front of your cluster, since Logstash will do that itself for all provided hosts.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 30, 2018, 1:16pm UTC](https://discuss.elastic.co/t/possible-to-loadbalance-logstash-via-dns/138133/5 "2018-07-30T13:16:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.