# Hot-Warm-Cold - avoid duplicates

**URL:** <https://discuss.elastic.co/t/hot-warm-cold-avoid-duplicates/226829>\
**Category:** Elasticsearch\
**Created:** [April 7, 2020, 6:08am UTC](https://discuss.elastic.co/t/hot-warm-cold-avoid-duplicates/226829 "2020-04-07T06:08:21Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Linuus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/linuus/32/65785_2.png) [@Linuus](https://discuss.elastic.co/u/Linuus)\
**Post date:** [April 7, 2020, 6:08am UTC](https://discuss.elastic.co/t/hot-warm-cold-avoid-duplicates/226829/1 "2020-04-07T06:08:22Z")

</div>

Hi!

I'm moving to a new ES cluster which also uses the Hot-Warm-Cold architecture. Now, when we imported our old data (600 million documents) it took quite a long time and after a while we noticed that we have duplicate documents, with the same `_id` field.

It seems like a few documents are present in both the "Hot" index as well as the "Warm" index. Previously this wasn't possible since we only had one index, but now there are separate indexes for Hot and Warm so duplicates are possible.

Are there any ways of avoiding these things happening?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2020, 6:08am UTC](https://discuss.elastic.co/t/hot-warm-cold-avoid-duplicates/226829/2 "2020-05-05T06:08:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.