Hi!
I have a lot of Filebeats (one per computer I want logs to be inserted in ElasticSearch)
Then I have a cluster of two Logstash nodes, both with the exact same configuration.
They have a disk persisted queue, 1 GB for it.
Filebeats outputs are configured like this:
output.logstash:
hosts: ["logstash1.host1:5044","logstash2.host2:5044"]
loadbalance: true
Sometimes a big logs file (about 11 million lines) is copied to a path a filebeat will manage it.
Elastic had a problem and stopped inserting data, so logstash queues filled up.
However, then one of the queues have been getting empty as Elastic was inserting documents, BUT the other queue does not get empty.
The question is, why does the queue of one Logstash get empty but not the other one?!?!?!?!
Are not logstash/filebeat smart enough to know one logtash queue is full and the other one empty?
Later on, the empty queue has filled again, I do not understand anything (I do not see in the kibana monitoring any of the logstash failing)
Thanks in advance!