# The python client avoids the \_parent field when it does reindex **URL:** \ **Category:** Elasticsearch\ **Created:** [October 12, 2014, 3:30pm UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207 "2014-10-12T15:30:32Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![costya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costya/32/751_2.png) [@costya](https://discuss.elastic.co/u/costya)\ **Post date:** [October 12, 2014, 3:30pm UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207/1 "2014-10-12T15:30:32Z")
Hi, When we run the following code: from elasticsearch import Elasticsearch from elasticsearch.helpers import reindex if **name** == " **main**": es = Elasticsearch() reindex(es, source\_index='2014\_03', target\_index='2014\_03\_new', chunk\_size =500, scroll='5m') We get the 2014\_03\_new index with all the fields but the \_parent field inserted right. However, the \_parent field, which we have in most of the documents of the original index is consistently missing in all of the documents of the new index. It looks like a bug in the client's code. We will appreciate your help. Regards, Costya, Totango Metrics. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Honza\_Kral](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honza_kral/32/876_2.png) [@Honza\_Kral](https://discuss.elastic.co/u/Honza_Kral)\ **Post date:** [October 12, 2014, 6:46pm UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207/2 "2014-10-12T18:46:26Z")
Hi Costya, the code actually looks for parent and should move it around. Are you sure you have your mappings set up correctly for the new index to include the parent/child relationship? Thanks On Sun, Oct 12, 2014 at 5:30 PM, Costya Regev [costya@totango.com](mailto:costya@totango.com) wrote: > Hi, > > When we run the following code: > > from elasticsearch import Elasticsearch > from elasticsearch.helpers import reindex > > if **name** == " **main**": > es = Elasticsearch() > reindex(es, source\_index='2014\_03', target\_index='2014\_03\_new', > chunk\_size=500, scroll='5m') > > We get the 2014\_03\_new index with all the fields but the \_parent field > inserted right. However, the \_parent field, which we have in most of the > documents of the original index is consistently missing in all of the > documents of the new index. > > It looks like a bug in the client's code. > > We will appreciate your help. > > Regards, > Costya, Totango Metrics. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CABfdDiob4EpAk1WgPKOV816G-myjHu5zJQLLgwPM5J1V65mJPQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CABfdDiob4EpAk1WgPKOV816G-myjHu5zJQLLgwPM5J1V65mJPQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![costya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costya/32/751_2.png) [@costya](https://discuss.elastic.co/u/costya)\ **Post date:** [October 13, 2014, 7:36am UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207/3 "2014-10-13T07:36:33Z")
Hi Kral, I think that the mapping is fine. Here it is: "mappings":{ "account":{ "properties":{ "name":{ "type":"string", "index":"not\_analyzed" } ``` ... ANOTHER ACCOUNT FIELDS ... }, "_routing":{ "required":true, "path":"name" } ``` }, "event":{ "properties":{ ``` ... SOME EVENT FIELDS ... "account":{ "type":"object", "properties":{ ... SOME MORE FIELDS ... "name":{ "type":"string", "index":"no" } } } }, "_parent":{ "type":"account" }, "_timestamp":{ "enabled":true, "store":true }, "_routing":{ "required":true, "path":"account.name" } ``` } } Do you see anything wrong here? Thx On Sunday, October 12, 2014 9:46:37 PM UTC+3, Honza Král wrote: > Hi Costya, > > the code actually looks for parent and should move it around. Are you > sure you have your mappings set up correctly for the new index to > include the parent/child relationship? > > Thanks > > On Sun, Oct 12, 2014 at 5:30 PM, Costya Regev \<[cos...@totango.com](mailto:cos...@totango.com) > \\> wrote: > > > Hi, > > > > When we run the following code: > > > > from elasticsearch import Elasticsearch > > from elasticsearch.helpers import reindex > > > > if **name** == " **main**": > > es = Elasticsearch() > > reindex(es, source\_index='2014\_03', target\_index='2014\_03\_new', > > chunk\_size=500, scroll='5m') > > > > We get the 2014\_03\_new index with all the fields but the \_parent field > > inserted right. However, the \_parent field, which we have in most of the > > documents of the original index is consistently missing in all of the > > documents of the new index. > > > > It looks like a bug in the client's code. > > > > We will appreciate your help. > > > > Regards, > > Costya, Totango Metrics. > > > > -- > > You received this message because you are subscribed to the Google > > Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/62eb9ecc-b88b-439e-bea7-b31141f1deb1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62eb9ecc-b88b-439e-bea7-b31141f1deb1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Honza\_Kral](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/honza_kral/32/876_2.png) [@Honza\_Kral](https://discuss.elastic.co/u/Honza_Kral)\ **Post date:** [October 13, 2014, 2:54pm UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207/4 "2014-10-13T14:54:20Z")
Hi, this is indeed a problem with elasticsearch, sorry it took me so long to realize this. The search api (and therefore the scan helper which is used by the reindex) doesn't return the value for \_parent by default. I created an issue in elasticsearch to change that (0). The workaround is to just use the scan and bulk methods (look how reindex looks, copy-paste and modify) to add fields=['\_source', '\_parent'] to the scan call and create your own version of expand\_action callback that will extract the \_parent value from the \_fields dictionary. This is very far from ideal so I also created an issue for elasticsearch-py (1) to enable doing this (maybe even do this by default) and describe the issues in the docs. Thanks for reporting this and sorry for the oversight, Honza 0 - [Return \_parent value by default for child documents · Issue #8068 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/8068) 1 - [Reindex doesn't support \_parent and \_routing · Issue #140 · elastic/elasticsearch-py · GitHub](https://github.com/elasticsearch/elasticsearch-py/issues/140) On Mon, Oct 13, 2014 at 9:36 AM, Costya Regev [costya@totango.com](mailto:costya@totango.com) wrote: > Hi Kral, > > I think that the mapping is fine. Here it is: > > "mappings":{ > "account":{ > "properties":{ > "name":{ > "type":"string", > "index":"not\_analyzed" > } > > ``` > ... ANOTHER ACCOUNT FIELDS ... > > }, > "_routing":{ > "required":true, > "path":"name" > } > > ``` > > }, > "event":{ > "properties":{ > > ``` > ... SOME EVENT FIELDS ... > > "account":{ > "type":"object", > "properties":{ > > ... SOME MORE FIELDS ... > > "name":{ > "type":"string", > "index":"no" > } > } > } > }, > "_parent":{ > "type":"account" > }, > "_timestamp":{ > "enabled":true, > "store":true > }, > "_routing":{ > "required":true, > "path":"account.name" > } > > ``` > > } > } > > Do you see anything wrong here? > Thx > > On Sunday, October 12, 2014 9:46:37 PM UTC+3, Honza Král wrote: > > > Hi Costya, > > > > the code actually looks for parent and should move it around. Are you > > sure you have your mappings set up correctly for the new index to > > include the parent/child relationship? > > > > Thanks > > > > On Sun, Oct 12, 2014 at 5:30 PM, Costya Regev [cos...@totango.com](mailto:cos...@totango.com) wrote: > > > > > Hi, > > > > > > When we run the following code: > > > > > > from elasticsearch import Elasticsearch > > > from elasticsearch.helpers import reindex > > > > > > if **name** == " **main**": > > > es = Elasticsearch() > > > reindex(es, source\_index='2014\_03', target\_index='2014\_03\_new', > > > chunk\_size=500, scroll='5m') > > > > > > We get the 2014\_03\_new index with all the fields but the \_parent field > > > inserted right. However, the \_parent field, which we have in most of the > > > documents of the original index is consistently missing in all of the > > > documents of the new index. > > > > > > It looks like a bug in the client's code. > > > > > > We will appreciate your help. > > > > > > Regards, > > > Costya, Totango Metrics. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b52dc918-4529-40ab-97e0-2a3b0176ec59%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/62eb9ecc-b88b-439e-bea7-b31141f1deb1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62eb9ecc-b88b-439e-bea7-b31141f1deb1%40googlegroups.com). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CABfdDiotzp4sPf5UDDBmD2wbpN92wAw\_CUCo2o0ptK7-Vk7G8w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CABfdDiotzp4sPf5UDDBmD2wbpN92wAw_CUCo2o0ptK7-Vk7G8w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:56am UTC](https://discuss.elastic.co/t/the-python-client-avoids-the--parent-field-when-it-does-reindex/20207/5 "2017-07-06T00:56:31Z")