How to re fetch data from Kafka, from the beginning of topic, without changing consumer group? (resetting consumer group offset)


I am using ELK GA 5.0.0. I have a logstash instance to pull data from a kafka topic, like below;

	group_id => "kafkagroupA"
	topics => ["kafkatopicA"]
	bootstrap_servers => ",,"
	codec => json

due to some config errors, my logs of past 2 days hasn't entered elasticsearch, but logstash was consuming. Now, I have corrected the config, and logs are coming fine. How can I get the old logs (past 2 days)? It is there in Kafka topic. I cant change the consumer group (group_id). Is there any way to achieve this?

Thanks in advance.

The offsets for the Consumer Group are stored in Kafka itself per partition.

You need to use Kafka tools to do this. Here is a link to an article that may help -

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.