Hello I'm facing an issue with duplicate records in my index on rollover. [Logstash duplication on rollover] I thought using rollover alias would let only one index to be "write" index which is supposed to not let any batch of records being indexed to be written in 2 indices. Also I have specif…

Any update here? It's an interesting problem. [image] SamehSaeed: That is correct, and there are no duplicates throughout the day. Only during the rollover, I find these duplicates. If the SQL query can return the same "record" multiple times, you are going to have a risk of that record (d…

Discuss the Elastic Stack

Duplicate records in Elasticsearch

Elastic Stack Logstash

Tortoise July 31, 2025, 1:09pm 6

Hello,

Is this issue similar to one discussed below :

Thanks!!

Topic		Replies	Views
Rollover Index duplication data,data coming from logstash Elasticsearch	19	1726	July 14, 2023
Duplication Due to rollover policy in kibana,data coming from logstash pipeline Logstash	10	782	January 13, 2023
Duplicates after index rollover Elasticsearch ilm-index-lifecycle-management	17	1000	October 26, 2023
Duplicate IDs across rollover indices Elasticsearch	6	2383	September 23, 2019
Duplicate records in Elasticsearch Logstash	3	424	March 5, 2021

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Duplicate records in Elasticsearch

Related topics