Kafka - Commit offset only after batch has been processed

Elastic Stack Logstash
1

Hey,

I understand after reading the code of the logstash-input-kafka plugin, that the offset it being committed after the batch has been transferred to the logstash internal queue, but not actually sent to the output (In my case Elasticsearch output).

Is there a wait to force logstash to wait for the batch to be fully committed to Elasticsearch before committing the offset to Kafka?

2

This is currently not possible, but there is an open issue covering discussions around this type of functionality.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.