# Elasticsearch reindex is not working

**URL:** <https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589>\
**Category:** Elasticsearch\
**Created:** [June 15, 2017, 4:01pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589 "2017-06-15T16:01:15Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 15, 2017, 4:01pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/1 "2017-06-15T16:01:15Z")

</div>

We have being trying to perform re indexing of existing to new index with change in mapping. But indexing is not happening properly.

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
green open \*\*\*\*\*\*\* iso4UtTXSwuDTNrIMZXjpw 5 1 408486 84 10.4gb 5.2gb  
green open \*\*\*\*\*\*\* wSxaFkrXQJujiGOWhtyEDQ 5 1 408486 201678 29.6gb 15.8gb

docs.count is getting matched but not with docs.deleted and is not getting exited from sense plugin

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 15, 2017, 5:02pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/2 "2017-06-15T17:02:39Z")

</div>

What is the source? What is the target in your example?

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 15, 2017, 5:35pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/3 "2017-06-15T17:35:58Z")

</div>

green open AAAA iso4UtTXSwuDTNrIMZXjpw 5 1 408486 84 10.4gb 5.2gb  
green open BBBB wSxaFkrXQJujiGOWhtyEDQ 5 1 408486 201678 29.6gb 15.8gb

we are re indexing AAAA to BBBB with new mapping

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 15, 2017, 6:00pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/4 "2017-06-15T18:00:05Z")

</div>

Issue in production.. We're not able re index it to new index with new mapping

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 15, 2017, 7:50pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/5 "2017-06-15T19:50:10Z")

</div>

So? What is the difference between old and new mapping?

Did you remove the BBBB index before launching the reindex operation?

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 16, 2017, 2:46am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/6 "2017-06-16T02:46:52Z")

</div>

We are adding date field on to the mapping and BBBB is new index  
First we created the mapping and then tried reindexing the AAAA to BBBB

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 16, 2017, 6:44am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/7 "2017-06-16T06:44:59Z")

</div>

Can you reproduce with a simple example and share the full script so we can better understand?

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 16, 2017, 11:29am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/8 "2017-06-16T11:29:00Z")

</div>

David when we execute re indexing why do doc.deleted get increased? Is it normal behavior?

get /\_cat/indices?v

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 16, 2017, 2:27pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/9 "2017-06-16T14:27:48Z")

</div>

> [@hello4usharath](#):
>
> David when we execute re indexing why do doc.deleted get increased?

That's what I want to understand actually. So that's the reason I asked for a full script which helps to understand what you are exactly doing. All the steps, please.

I suspect that you are doing updates of existing documents, not reindexing to a fresh and clean index. That's why I'm asking.

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 16, 2017, 4:30pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/10 "2017-06-16T16:30:03Z")

</div>

before this we create ticket\_v2 mapping and run below command

POST \_reindex  
{  
"source": {  
"index": "ticket\_v1"  
},  
"dest": {  
"index": "ticket\_v2"  
}  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 16, 2017, 5:47pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/11 "2017-06-16T17:47:17Z")

</div>

So you don't delete `ticket_v2`?

---

<div class="post-metadata">

**Author:** ![lwintergerst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lwintergerst/32/18164_2.png) [@lwintergerst](https://discuss.elastic.co/u/lwintergerst)\
**Post date:** [June 16, 2017, 6:05pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/12 "2017-06-16T18:05:52Z")

</div>

Could you also post the exact command that you are using for the reindex? Feel free to black out the index name

Which version of Elasticsearch are you running? There have been a few bugs in the past

If you need a quick fix I would consider to expunge the deleted documents using the force merge API  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html)  
POST /twitter/\_forcemerge?only\_expunge\_deletes=true

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 16, 2017, 6:26pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/13 "2017-06-16T18:26:05Z")

</div>

We are not deleting ticket\_v2. While re indexing to ticket\_v2 doc.count matches with ticket\_v1 but doc.deleted doesn't and count gets increased by 4 to 5 times. If you see first post wherein first line doc.deleted is 84 and in second line nearly 20 thousand.  
What could be the issue?

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 16, 2017, 6:32pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/14 "2017-06-16T18:32:27Z")

</div>

why to do force merge it? we want to re index the data to new index with new mapping.

while performing the re indexing docs.count gets matched but only concern is docs.deleted number varies for long time . That is concern

we are using 5.2.0 version of elastic search.

POST \_reindex  
{  
"source": {  
"index": "ticket\_v1"  
},  
"dest": {  
"index": "ticket\_v2"  
}  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 16, 2017, 7:21pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/15 "2017-06-16T19:21:47Z")

</div>

So upgrade is the first thing to do.  
Then create an empty target index. Which means DELETE target.  
Then launch a single run of reindex from source to target and share the results here.

---

<div class="post-metadata">

**Author:** ![hello4usharath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello4usharath/32/19175_2.png) [@hello4usharath](https://discuss.elastic.co/u/hello4usharath)\
**Post date:** [June 17, 2017, 5:25am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/16 "2017-06-17T05:25:15Z")

</div>

What makes to think we need upgrade?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 17, 2017, 7:21am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/17 "2017-06-17T07:21:33Z")

</div>

This:

> [@Elasticsearch reindex is not working](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/12):
>
> Could you also post the exact command that you are using for the reindex? Feel free to black out the index name Which version of Elasticsearch are you running? There have been a few bugs in the past If you need a quick fix I would consider to expunge the deleted documents using the force merge API [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-forcemerge.html) POST /twitter/\_forcemerge?only\_expunge\_deletes=true

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 15, 2017, 7:21am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-not-working/89589/18 "2017-07-15T07:21:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
