# ES design regarding duplicates across indexes

**URL:** <https://discuss.elastic.co/t/es-design-regarding-duplicates-across-indexes/117852>\
**Category:** Elasticsearch\
**Created:** [January 31, 2018, 4:24pm UTC](https://discuss.elastic.co/t/es-design-regarding-duplicates-across-indexes/117852 "2018-01-31T16:24:18Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 1, 2018, 7:39am UTC](https://discuss.elastic.co/t/es-design-regarding-duplicates-across-indexes/117852/2 "2018-02-01T07:39:08Z")

</div>

Using time based is best.

> [@seanwang30](#):
>
> The problem is that if a service exists across the hours, say index1 and index2 and we ask for 2 hours of data, the query would return duplicates coming from both indices.

Can you elaborate more on your document structure? If this is time based data then usually you'd have an `event` at `$time`, and then `event` at `$time+n` and they would both have different values. Is that not the case?

> [@Efficient merging of index data stored in heterogeneous repositories](https://discuss.elastic.co/t/efficient-merging-of-index-data-stored-in-heterogeneous-repositories/17852/1):
>
> My setup looks like this: 2 master nodes

That's bad, see [Important Configuration Changes | Elasticsearch: The Definitive Guide [2.x] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/important-configuration-changes.html#_minimum_master_nodes)

---

_[View the full topic](https://discuss.elastic.co/t/es-design-regarding-duplicates-across-indexes/117852)._