# Upgrading 1.7 -\> 5.0 with reindex-from-remote and parent/child docs

**URL:** <https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463>\
**Category:** Elasticsearch\
**Created:** [October 20, 2016, 5:00am UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463 "2016-10-20T05:00:29Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![nkw](https://avatars.discourse-cdn.com/v4/letter/n/e36b37/32.png) [@nkw](https://discuss.elastic.co/u/nkw)\
**Post date:** [October 20, 2016, 5:00am UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/1 "2016-10-20T05:00:29Z")

</div>

I'm attempting to move an index from a 1.7 cluster to a 5.0-rc1 cluster via the reindex-from-remote procedure [documented here](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/reindex-upgrade.html#reindex-upgrade-remote) and [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/docs-reindex.html#reindex-from-remote).

One type of document reindex works just fine (the parent documents), however a second type which contains a \_parent mapping to the first type fails. The reindex task errors with failures of type "routing\_missing\_exception". (Since the mapping has a \_parent field, I believe the \_routing is set to 'required'). It appears neither the \_parent or \_routing fields are being set. After reviewing the documentation I attempted:

1. Trying all the available settings for the `routing` option on the dest request e.g. `keep/discard/=<some text>`

2. Creating a script to manually assign the \_routing field from the \_parent field. (Grasping at straws here.) (e.g. `"ctx._routing = ctx._parent"`).

3. Adding a `"fields": "*"` to a `"query"` on the source.

(1) and (2) did nothing, and I received the same error. (3) errored with `"Deprecated field [fields] used, expected [stored_fields] instead"`, but if I then use `stored_fields` I get `"Validation Failed: 1: stored_fields is in not supported in this context;"`

The docs say:

> By default if \_reindex sees a document with routing then the routing is preserved unless it’s changed by the script.

I suspect that is technically true, but \_reindex is not seeing the \_parent or \_routing from the source.

I looked through the source and it appears the reindex-from-remote uses the scan/scroll method, however on 1.7, I have been unable to get a scan/scroll call to return either \_parent or \_routing in the results.

I even noticed there were tests for parent/child documents from remote in the \_reindex source, so I'm thinking I just haven't figured out the right way to do it. What am I missing?

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 20, 2016, 3:41pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/2 "2016-10-20T15:41:00Z")

</div>

Those tests don't cover reindex-from-remote from an old version. I tested a bunch of that stuff manually but I never tested with parent/child. I'm not sure there _is_ a workaround given the rename from `fields` to `stored_fields`. I think this highlights the need to have automated testing for the versions we support for reindex from remote. The trouble is that all the fancy code we use to spin up an external cluster for testing needs stuff we added in 5.0. The other backwards compatibility tests all use static files generated by some version of Elasticsearch.

I'll look at this further so I opened

> <https://github.com/elastic/elasticsearch/issues/21044>

I'll try to provide a workaround in the issue if I find one. At this point I don't expect any fix for this to go into 5.0 so if there isn't a workaround you might be stuck waiting for 5.1 or 5.0.1.

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 20, 2016, 9:16pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/3 "2016-10-20T21:16:08Z")

</div>

I've renamed the issue because I isolated the problem to 1.7. It has the same number but it no longer has the same name so gmail threading issue may occur.

---

<div class="post-metadata">

**Author:** ![nkw](https://avatars.discourse-cdn.com/v4/letter/n/e36b37/32.png) [@nkw](https://discuss.elastic.co/u/nkw)\
**Post date:** [October 21, 2016, 6:26pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/4 "2016-10-21T18:26:54Z")

</div>

@nik9000 Just saw your fix get merged. Thanks! You rock!

> <https://github.com/elastic/elasticsearch/commit/18393a06f373a2132972d5551bc9ed3ae28022e6>

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 21, 2016, 6:29pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/5 "2016-10-21T18:29:17Z")

</div>

Thanks! I'm backporting to 5.x at the moment. I don't think this'll make 5.0.0 but it'll make 5.0.1.

---

<div class="post-metadata">

**Author:** ![nkw](https://avatars.discourse-cdn.com/v4/letter/n/e36b37/32.png) [@nkw](https://discuss.elastic.co/u/nkw)\
**Post date:** [October 25, 2016, 7:48pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/6 "2016-10-25T19:48:32Z")

</div>

As a documentation issue, if the fix isn't going to make it into 5.0.0, is it appropriate to note the issue with parent/child documents and 1.x -\> 5.0 reindex-from-remote in the documentation located both [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/setup-upgrade.html) and [here](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/reindex-upgrade.html#reindex-upgrade-remote)?

My worry is there might be a lot of people doing a 1.x to 5.0 upgrade if they had to skip 2.0 due to the "dots in fields" issues (notwithstanding 2.4). Right now the docs callout reindex-from-remote as the only way to do that short of an intervening migration to 2.4. I was thinking it might be a good idea to present a warning regarding the parent/child issue if it is going to there at 5.0 release (or perhaps also suggest people with that issue wait until 5.0.1).

I'm happy to submit a PR to the docs if you think it is a good idea.

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [October 25, 2016, 8:00pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/7 "2016-10-25T20:00:23Z")

</div>

> [@nkw](#):
>
> I'm happy to submit a PR to the docs if you think it is a good idea.

I would merge it with glee.

And, yes, I expect to merge the fix for 5.0.1.

---

<div class="post-metadata">

**Author:** ![nkw](https://avatars.discourse-cdn.com/v4/letter/n/e36b37/32.png) [@nkw](https://discuss.elastic.co/u/nkw)\
**Post date:** [October 26, 2016, 4:36pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/8 "2016-10-26T16:36:55Z")

</div>

Not sure if I targeted the right branch, but FWIW:

> <https://github.com/elastic/elasticsearch/pull/21120>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:09pm UTC](https://discuss.elastic.co/t/upgrading-1-7-5-0-with-reindex-from-remote-and-parent-child-docs/63463/9 "2017-07-05T22:09:23Z")

</div>


