# Intersect documents with same id in two indices (5.6)

**URL:** <https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321>\
**Category:** Elasticsearch\
**Created:** [July 15, 2020, 3:21pm UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321 "2020-07-15T15:21:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AsafHirschfeld](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AsafHirschfeld](https://discuss.elastic.co/u/AsafHirschfeld)\
**Post date:** [July 15, 2020, 3:21pm UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321/1 "2020-07-15T15:21:30Z")

</div>

I have two indices with documents, each document assigned with a unique id.

My goal is to find duplicate documents among the indices.

Is there any sort of an intersection aggregation that i can make in order to determine whether my indices share common id documents?

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [July 15, 2020, 6:52pm UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321/2 "2020-07-15T18:52:59Z")

</div>

You could use transform, we have an example in the docs for [comparing 2 indices](https://www.elastic.co/guide/en/elasticsearch/reference/7.8/transform-painless-examples.html#painless-compare).

---

<div class="post-metadata">

**Author:** ![AsafHirschfeld](https://avatars.discourse-cdn.com/v4/letter/a/65b543/32.png) [@AsafHirschfeld](https://discuss.elastic.co/u/AsafHirschfeld)\
**Post date:** [July 16, 2020, 9:06am UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321/3 "2020-07-16T09:06:40Z")

</div>

I forgot to mention that im using elastic 5.6

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [July 16, 2020, 9:55am UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321/4 "2020-07-16T09:55:37Z")

</div>

ok, 5.6 is fairly old and EOL since 2019-03-11, please check if an upgrade is possible:

If you can't go to `7.x`:

- `>= 6.1` would give you access to composite aggs, which would let you page through aggs, in a nutshell you could write your own transform
- `>= 6.8` would allow you to use transform + CCS from a `>= 7.6` cluster that queries your `>=6.8` cluster

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 13, 2020, 9:56am UTC](https://discuss.elastic.co/t/intersect-documents-with-same-id-in-two-indices-5-6/241321/5 "2020-08-13T09:56:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.