# Compare 2 indices and find missing documents

**URL:** <https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434>\
**Category:** Elasticsearch\
**Tags:** reindex\
**Created:** [August 23, 2023, 7:12am UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434 "2023-08-23T07:12:12Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![arks1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arks1/32/123339_2.png) [@arks1](https://discuss.elastic.co/u/arks1)\
**Post date:** [August 23, 2023, 7:12am UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434/1 "2023-08-23T07:12:12Z")

</div>

I have 2 indices, `index_a` and `index_b`.

The 2 indices have documents with the _almost_ the exact same template. `index_b` has some extra fields which was introduced as part of a new feature, but it also has all the fields already present in `index_a`.

We have noticed that that `index_b` is missing about 2000 documents which are present in `index_a`. We found this out by using the `_count` API.

Now the question is, **is there a way to find out the actual missing documents? Only the missing Ids should also be enough for a start.**

Both the indices have a field called `member_id` which is unique for each document and is the same as the document id, so retrieving only the missing id fields should also be enough.

I cannot compare the index directly to the source database because this data comes from an external API.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 23, 2023, 10:59am UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434/2 "2023-08-23T10:59:00Z")

</div>

You have to do that "manually".

Search for all documents in the first index, get the ids and then run a multiget API call to check.

---

<div class="post-metadata">

**Author:** ![arks1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/arks1/32/123339_2.png) [@arks1](https://discuss.elastic.co/u/arks1)\
**Post date:** [August 24, 2023, 7:42am UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434/3 "2023-08-24T07:42:09Z")

</div>

Thank you for the response. I got a list of ids from the first index. When I try to use the `_mget` API, I can see for the ids that do not exist in `index_b`, the response body contains the following response.

```json
{
    "_index": "tm_member5",
    "_id": "0",
    "found": false
}

```

I am going to write some logic to be able to extract the ids from the objects where `found == false`

But just wanted to know if the \_mget api itself has any input parameter I can use to do that. I looked at the doc but could not find anything of this sort.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [August 24, 2023, 4:57pm UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434/4 "2023-08-24T16:57:29Z")

</div>

I'd use jq for this. It helps to filter a node and print whatever you need.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 21, 2023, 4:57pm UTC](https://discuss.elastic.co/t/compare-2-indices-and-find-missing-documents/341434/5 "2023-09-21T16:57:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.