I use two logstash server with two difference group_id in input.kafka to get log data from kafka-input and ,but they become a same consumer and average the log data.
What the error I am not discover?
This is logstash-1(5.0.0) configuration:
input { kafka { bootstrap_servers => "host1:9092,host2:9092,host3:9092" group_id => "newlog" topics => ["online"] client_id => "newlog" auto_offset_reset => earliest consumer_threads => 3 } }
This is logstash-2(1.5.4) configuration:
input { kafka { zk_connect => "cstr-01:2181,cstr-02:2181,cstr-03:2181" group_id => "logstash" topic_id => "online" reset_beginning => false consumer_threads => 3 queue_size => 50000 rebalance_max_retries => 4 } }
Best regards!