Logstash integration Kafka: Support, or document support for, setting consumer isolation.level to read_committed

Hi; we're using https://github.com/logstash-plugins/logstash-integration-kafka to transcribe Kafka Avro message fields from Kafka to an Elastic search index for our reports.

Recently we noticed that the consumer by default uses read_uncommitted, which when combined with a transactional producer lead to fields being transcribed for messages that were later rolled back, messing up our reports.

There seem to be quite a few overrideable properties already, yet I could not find anything corresponding to: https://kafka.apache.org/25/documentation/#isolation.level

Is there any way to set the consumer isolation level for Kafka to read_committed? Could it be added or documented?

I've since added a pull request with what I think are the required changes, although I'm not too familiar with the code or Ruby (yet) : https://github.com/logstash-plugins/logstash-integration-kafka/pull/44

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.