Elasticsearch output - load balancing selection

kobyb · September 29, 2016, 7:04pm

Say I have three logstash servers loadbalanced behind and F5. All three servers are identical, and contain an elasticsearch output plugin with a host array of something like: hosts => ["host1","host2",host3"].

How is the output load balanced? Round Robin, least used, etc? What I'm trying to figure out is, if the three logstash servers would get in a rhythm of rotating through the same hosts at the same time, thus swamping one, while the others go unused.

Thanks

magnusbaeck · October 5, 2016, 6:01am

The way I read the code (below) each Logstash instance will distribute the requests evenly between the hosts and for hosts with an equal number of posted requests it'll pick one of them at random. So I guess least used followed by round robin as a tie-breaker.

github.com

logstash-plugins/logstash-output-elasticsearch/blob/v5.1.2/lib/logstash/outputs/elasticsearch/http_client/pool.rb#L345-L371


def get_connection
  @state_mutex.synchronize do
    # The goal here is to pick a random connection from the least-in-use connections
    # We want some randomness so that we don't hit the same node over and over, but
    # we also want more 'fair' behavior in the event of high concurrency
    eligible_set = nil
    lowest_value_seen = nil
    @url_info.each do |url,meta|
      meta_in_use = meta[:in_use]
      next if meta[:dead]


      if lowest_value_seen.nil? || meta_in_use < lowest_value_seen
        lowest_value_seen = meta_in_use
        eligible_set = [[url, meta]]
      elsif lowest_value_seen == meta_in_use
        eligible_set << [url, meta]
      end
    end


    return nil if eligible_set.nil?

This file has been truncated. show original

Topic		Replies	Views
Load balance output between multiple hosts Logstash	10	3012	August 15, 2018
Confirmation about the load balance mechanism when using elasticsearch output Logstash	3	509	December 15, 2016
Logstash output for multiple elasticsearch instance Logstash	9	32252	July 6, 2017
How do I confirm DNS round robin is working correctly? Logstash	8	1797	May 11, 2017
Output to an Elasticsearch cluster - Details? Logstash	1	256	August 21, 2019

Elasticsearch output - load balancing selection

Related topics