# Problem indexing ES with batches

**URL:** <https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961>\
**Category:** Elasticsearch\
**Created:** [September 9, 2020, 6:48am UTC](https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961 "2020-09-09T06:48:20Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![magorbe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magorbe/32/36871_2.png) [@magorbe](https://discuss.elastic.co/u/magorbe)\
**Post date:** [September 9, 2020, 6:48am UTC](https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961/1 "2020-09-09T06:48:20Z")

</div>

Hello. I am doing reindexing in ES with the next command :

nohup curl -XPOST http:/bla:9200/\_reindex -d '{ "source": { "index": "origin\_index", "size": 5000 },  
"dest": {"index": "dest\_index" }}' \> reindex\_100\_21.out 2\> reindex\_100\_21.err

We have 25.853.770 documents in origin\_idex, and the problem is that We have 25.849.990 in dest\_index and no errors in the log... seems that We are loosing the last batch...

Is there any option to indicate this ? Thanks you very much.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 10, 2020, 8:11pm UTC](https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961/2 "2020-09-10T20:11:09Z")

</div>

What is the refresh interval of the destination index? Did you issue a refresh before counting?

---

<div class="post-metadata">

**Author:** ![magorbe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magorbe/32/36871_2.png) [@magorbe](https://discuss.elastic.co/u/magorbe)\
**Post date:** [September 11, 2020, 7:36am UTC](https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961/3 "2020-09-11T07:36:29Z")

</div>

I relaunch the reindex now with another size of batch

nohup curl -XPOST [http://ip:9200/\_reindex](http://ip:9200/_reindex) -d '{ "source": { "index": "origin", "size": 50000 }, "dest": {"index": "dest", "op\_type": "create" }}' &\> reindex\_50000.out

But now, i can see this error:

{"took":"2h","timed\_out":false,"total":25853780,"updated":0,"created":25700000,"batches":514,"version\_conflicts":0,"noops":0,"retries":0,"failures":[{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [32]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [43]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [42]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [33]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [34]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [44]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [36]"}},{"shard":-1,"index":null,"reason":{"type":"search\_context\_missing\_exception","reason":"No search context found for id [37]"}}]}

So, that's the reason why We are loosing batches... It's possible we need more timeout\_time... Is this possible ? Thanks !!!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2020, 7:36am UTC](https://discuss.elastic.co/t/problem-indexing-es-with-batches/247961/4 "2020-10-09T07:36:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.