# Elasticsearch compare two indices

**URL:** <https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181>\
**Category:** Elasticsearch\
**Created:** [March 30, 2018, 5:20am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181 "2018-03-30T05:20:36Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [March 30, 2018, 5:20am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/1 "2018-03-30T05:20:36Z")

</div>

How does one go about comparing two (or more) indices in elasticsearch? especially based on specific fields (not entire document).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2018, 8:07am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/2 "2018-03-30T08:07:29Z")

</div>

You can't do anything like a join.

You need to solve that at index time or on the client side IMO.

---

<div class="post-metadata">

**Author:** ![alexus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexus/32/12696_2.png) [@alexus](https://discuss.elastic.co/u/alexus)\
**Post date:** [March 30, 2018, 8:13pm UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/3 "2018-03-30T20:13:10Z")

</div>

just to reiterate; among of other fields, each of index's docs has at least following fields: `city`, `state` and `zip`.

like following:

index-0: \_id, retailName, city, state, zip, etc  
index-1: \_id, otherName, city, state, zip, etc  
index-2: \_id, otherName, city, state, zip, etc  
index-3: \_id, otherName, city, state, zip, etc

My goal is to compare docs (from index-1-3) with fields (city, state, zip) against index-0 and find how much percent wise of each index (1-3) are matches against index-0.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [March 30, 2018, 8:57pm UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/4 "2018-03-30T20:57:37Z")

</div>

You can't compare documents. You can search for documents or aggregate some fields from the resultset but you can't join documents to compare them which is I think what you asked for...

---

<div class="post-metadata">

**Author:** ![Frederik\_Mortensen](https://avatars.discourse-cdn.com/v4/letter/f/c6cbf5/32.png) [@Frederik\_Mortensen](https://discuss.elastic.co/u/Frederik_Mortensen)\
**Post date:** [April 24, 2018, 11:19am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/5 "2018-04-24T11:19:11Z")

</div>

Hi @dadoonet @alexus.

First of all I am new to the forum. And new to elastic. But I used the search function and found this thread. So I decided not to open yet another thread. I am not sure wether my situation is quite the same as alexus situation (regarding the definition of "document"). But I am having a similar problem. I am synchronizing from MySQL to Elastic Index1. Later I changed to Index2. Same MySQL table. Same index with same fields. Now I would like to compare the two indices. I know that many entries should be in both indices. But some are just in index 1, some are in index2 and some entries differ in their content between the indices. Is there a way to do this in Elastic?

Thanks for your help

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 24, 2018, 11:43am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/6 "2018-04-24T11:43:30Z")

</div>

I don't think there is an easy way to do that.

---

<div class="post-metadata">

**Author:** ![Frederik\_Mortensen](https://avatars.discourse-cdn.com/v4/letter/f/c6cbf5/32.png) [@Frederik\_Mortensen](https://discuss.elastic.co/u/Frederik_Mortensen)\
**Post date:** [April 24, 2018, 11:53am UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/7 "2018-04-24T11:53:16Z")

</div>

OK thanks for the fast answer @dadoonet. I think this could be a useful addition to Elasticsearch. Maybe you could add it as a feature request? And how would you try to compare the indices now? With scrolling and using an external script like java? Or simply replace the indices with full takes? Its quite a big index and takes some hours to synchronize it.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 24, 2018, 12:12pm UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/8 "2018-04-24T12:12:37Z")

</div>

I don't think it will be supported unless we support distributed joins which we don't.

> With scrolling and using an external script like java?

Yes. most likely.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2018, 12:12pm UTC](https://discuss.elastic.co/t/elasticsearch-compare-two-indices/126181/9 "2018-05-22T12:12:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
