Two Logstash nodes. Same config. Persistent queue filling only in one of them

Elastic Stack Logstash
1

Hello!

I have a 3 node Elasticsearch cluster, 2 Logstash nodes and about 100 filebeats sending data to Logstash. Every piece is 7.17
Both Logstash nodes have the exact same configuration. There is a 16 GB persistent queue on each node.
All Filebeats send data to both Logstash nodes using their internal Load Balancer (I mean all the filebeat.yml config files have this: loadbalance: true)
Both Logstash nodes send data to all three Elasticsearch nodes, I mean everything is balanced
When too many events arrive to Elasticsearch from Logstash (more than around 10 K events / sec), persistent queues are there to fill,
however only the node 1 persistent queue fills
Is this the expected behaviour? We upgraded from ELK 7.5 to 7.17, when we had 7.5 both queues filled.
Both Logstash send events to Elasticsearch, and when events arrive slower, persistent queue drains

What could be wrong in my ELK Stack?

2

Can you share your filebeat.yml ?

Also, which version of Filebeat are you using? Did you upgrade filebeat from 7.5 to 7.17 as well? What is the patch release of 7.17 that you are running?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.