Logstash read from kafka in a round-robin way


(Chengcheng Pei) #1

I have 3 partitions: 0, 1, 2. So the messages can be classified as 0, 1, 2.

eg:

1 message in partition 0: 0

3 messages in partition 1: 111

2 messages in partition 2: 22

How to make the logstash to consume messages in the order 012x12x1x
(x means no messages at that time). The order of consumed messages look
like: 012121.

There is partition.assignment.strategy in Kafka consumer configs (http://kafka.apache.org/documentation.html#consumerconfigs). I am looking for some plugins which implements this config to read data from kafka, reorder them, and write to kafka again.

Any idea?


(system) #2