I have an ELK cluster running in Docker Swarm mode. The cluster meed my needs because i have two replicas of elastic, but i have a problem with logstash running multiple instances.
When i send amount of logs to logstash, only one replica ingest data in elastic like bellow:
instance1 - 2 events ingested
instance2 - 100k events ingested
I allready try to use in memory queue and persistent queue, but get same problem.
Anyone have a idea for this problem? I'll need to use a kafka/redis in front of logstash?
my logstash service in compose:
command: logstash -f /usr/share/logstash/pipeline/logstash.conf
constraints: [node.role == worker]