Logstash Bulk load balancing

Elastic Stack Logstash
1

I have 12 Logstash Indexers (Reading off of kafka) and have been noticing that sometimes only one Data node is processing all of the bulk requests.

Is there something missing in my logstash configurations

I guess I could just choose 2 servers for each indexer to send data too rather then specifying the whole cluster but I was hopping the Load balancing algorithm was better.

	elasticsearch {
		hosts => [ {% for host in groups['es-data']|shuffle %}"{{ host }}:9210"{% if not loop.last %},{% endif %}{% endfor %} ]
		index => "%{dst_index}-%{+YYYY.MM.dd}"
		template => "{{ template_filename }}"
		template_overwrite => "true"
		workers => 32
                flush_size => 8000
		idle_flush_time => 1
		#sniffing => true 
    }```
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.