I want to have HA in Logstash, my scenario is:
N-Flume --> Kafka --> N Logstash --> ElasticSearch
I don't know if there are any recommendation about HA, I read https://www.elastic.co/guide/en/logstash/current/deploying-and-scaling.html about this topic. What I want to it's to have more many machines running logstash so if I lost one of machine I wouldn't lose the service.
When I execute more than one logstash instances with a "stdout { codec => rubydebug }}" I can't only see outputs in one console while the others I got:
"Settings: Default pipeline workers: 8
Pipeline main started"
Does it mean that only one instance gets data from Kafka?
I went down one logstash instance and suddenly the another instance started it to print traces of consumed events.