# How to reindex doc with multifield? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 20, 2016, 10:20am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964 "2016-07-20T10:20:27Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Moshe\_Sucaz](https://avatars.discourse-cdn.com/v4/letter/m/0ea827/32.png) [@Moshe\_Sucaz](https://discuss.elastic.co/u/Moshe_Sucaz)\ **Post date:** [July 20, 2016, 10:20am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/1 "2016-07-20T10:20:27Z")
Hi, I am writing a python script to reindex our files with new mappings on elastic 1.5.2 (before upgrade to 2.3). one of the changes was mappings of title: "title":{"type":"string"} that was changed to new mappings: "title": { "type": "string", "fields": { "english": { "type": "string", "analyzer": "buzzilla\_english" }, "arabic": { "type": "string", "analyzer": "buzzilla\_arabic" } } } I tried to copy each doc and change it to the new mapping as follow: old\_doc = hit['\_source'] new\_doc = old\_doc title = new\_doc['title'] language="english" //for the example new\_doc['title'] = {language:title } When running it it trying to "put" the new file, I get: {"error":"MapperParsingException[failed to parse [title]]; nested: ElasticsearchIllegalArgumentException[unknown property [english]]; ","status":400} I also tried: new\_doc['title'] = {} new\_doc['title']['fields'] = {language:title } But it also failed: {"error":"MapperParsingException[failed to parse [parent\_title]]; nested: ElasticsearchIllegalArgumentException[unknown property [fields]]; ","status":400} What is the correct way to do it please? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 20, 2016, 10:58am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/2 "2016-07-20T10:58:46Z")
Hi @Moshe_Sucaz, it's much easier than that. You just do: ```auto old_doc = hit['_source'] new_doc = old_doc ``` in other words, you don't need to alter the document. You just index the same document again based the new mapping. You cannot explicitly index [multi fields](https://www.elastic.co/guide/en/elasticsearch/reference/1.5/mapping-core-types.html#_multi_fields_3). They are automatically added by Elasticsearch and you can use them in queries. Daniel ---
**Author:** ![Moshe\_Sucaz](https://avatars.discourse-cdn.com/v4/letter/m/0ea827/32.png) [@Moshe\_Sucaz](https://discuss.elastic.co/u/Moshe_Sucaz)\ **Post date:** [July 20, 2016, 11:39am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/3 "2016-07-20T11:39:26Z")
Thanks Daniel, but in this way the title is not getting the analyzer I want. Where I wrote “language="english" //for the example” - I meant that I do manipulation there: The old doc has language “\_analyzer” field, as in elasticsearch version 1.5.2 and older versions. So for each doc I need to check its “\_analyzer” and set the title (and some more fields that we use analyzer for them) with the appropriate field. For example, If I read now a doc with Hebrew language, then I wanted the new doc to have title.hebrew with the old title and with our buzzilla\_hebrew analyzer. With the way you suggested I see the title as string (when opening with sense: "title": "Hello world”), and not as object, as I understand it should be: “title.hebrew": "Hello world” Also, when searching in this way: GET /[index name]/\_search { "query": { "multi\_match": { "type": "most\_fields", "query": "Hello world", "fields": ["title", "title.hebrew"] } } } I do get results, but when searching with "fields": ["title.hebrew"], I don't get results. How can I know that the doc will be analyzed with our buzzilla\_hebrew analyzer when I search for hebrew words in it? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 20, 2016, 11:55am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/4 "2016-07-20T11:55:01Z")
Hi @Moshe_Sucaz, so if I've understood you correctly, you have currently strings in different languages in the same field (e.g. for doc[0] the "title" field contains an English title, for doc[357] the "title" field contains an Italian title)? If that is the case, then you should map this differently. See the chapter [Getting Started with Languages](https://www.elastic.co/guide/en/elasticsearch/guide/current/language-intro.html) in the Definitive Guide. Daniel ---
**Author:** ![Moshe\_Sucaz](https://avatars.discourse-cdn.com/v4/letter/m/0ea827/32.png) [@Moshe\_Sucaz](https://discuss.elastic.co/u/Moshe_Sucaz)\ **Post date:** [July 20, 2016, 12:04pm UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/5 "2016-07-20T12:04:17Z")
Yes! That is exactly what I am trying to do. I did that as I understand from the link you sent. In the next page of this link ([https://www.elastic.co/guide/en/elasticsearch/guide/current/using-language-analyzers.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/using-language-analyzers.html)) I saw the use of multifields to index the title field twice: once with the english/hebrew/italian analyzer and once with the standard analyzer. But I still don't understand what am I doing wrong 😕 ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 20, 2016, 12:42pm UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/6 "2016-07-20T12:42:58Z")
Hi @Moshe_Sucaz, Ok, then let's walk through a concrete example so we're on the same page: Suppose you have two documents in your source index (let's call it /src): doc[1].title = "Hello World" doc[2].title = "Hallo erstmal" Suppose you also know that the language of `doc[1].title` is English and the language of `doc[2].title` is German. Now you create your target index (just in case: delete it if you run this example multiple times): ```auto DELETE /target PUT /target { "mappings": { "titles": { "properties": { "title": { "type": "string", "fields": { "english": { "type": "string", "analyzer": "english" }, "german": { "type": "string", "analyzer": "german" } } } } } } } ``` You reindex the documents as is, i.e. you don't change anything: ```auto PUT target/titles/1 { "title": "Hello World" } PUT target/titles/2 { "title": "Hallo erstmal" } ``` If you need to remember that doc id 1 is in English and doc id 2 is in German, so you need some kind of "language" field. Note that the "title" field (of each document) is now indexed with three different analyzers: 1. the "root" title field is analyzed with the "standard" analyzer 2. the "english" subfield is analyzed with the "english" analyzer 3. the "german" subfield is analyzed with the "german" analyzer So all analyzers run for each document you'll index. Now we can run a query (and as far as I've understood this concrete case did not yield a result for you): ```auto GET /target/titles/_search { "query": { "multi_match": { "type": "most_fields", "query": "Hello world", "fields": [ "title.german" ] } } } ``` which gives: ```auto { "took": 3, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 0.2712221, "hits": [ { "_index": "target", "_type": "titles", "_id": "1", "_score": 0.2712221, "_source": { "title": "Hello World" } } ] } } ``` because the `title.german` field also exists for the document that is actually in English. I hope we can get down to your problem based on this example. Daniel ---
**Author:** ![Moshe\_Sucaz](https://avatars.discourse-cdn.com/v4/letter/m/0ea827/32.png) [@Moshe\_Sucaz](https://discuss.elastic.co/u/Moshe_Sucaz)\ **Post date:** [July 21, 2016, 5:36am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/7 "2016-07-21T05:36:29Z")
Daniel, thank you very much for this detailed example! Now I understand it much better, and its working also for me like a magic... I didn't get results for "fields": ["title.hebrew"] because I accidentally run this query on the old index Let me please ask one small question regarding the reindex: I have a field with dots in its name. For this one I did run next lines in my script: new\_doc = old\_doc if 'foo.bar' in new\_doc: new\_doc['foo\_bar'] = new\_doc['foo.bar'] del new\_doc['foo.bar'] I just want to make sure I am doing it right... Thanks! ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 21, 2016, 6:08am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/8 "2016-07-21T06:08:27Z")
Hi Moshe, glad I could help finally resolve your problem. 🙂 To your question: Yes, this looks ok to me. Maybe you can come with an even more generic approach like something along these lines (pseudocode!): ```auto new_doc = {} for k, v in old_doc.items(): new_doc[k.replace_all(".", "_")] = v ``` I figure you are migrating to Elasticsearch 2.x where we disallowed dots in field names. I just wanted to make you aware that we allow them again in 5.0: See [https://github.com/elastic/elasticsearch/issues/15951](https://github.com/elastic/elasticsearch/issues/15951) and specifically [https://github.com/elastic/elasticsearch/issues/19443](https://github.com/elastic/elasticsearch/issues/19443). I don't know about your timeline but maybe you want to consider to upgrade to 5.0 (although this is a huge step and requires probably a lot of testing). But be aware that it is not yet final and we also don't have a release date. I just wanted to raise your awareness as you're struggling with removing dots from your field names. With 2.x we have also introduced a reindex feature in Elasticsearch. So if you change your mapping and don't need to do any transformations (unlike your case) you can use that also in the future (see [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html)). Daniel ---
**Author:** ![Moshe\_Sucaz](https://avatars.discourse-cdn.com/v4/letter/m/0ea827/32.png) [@Moshe\_Sucaz](https://discuss.elastic.co/u/Moshe_Sucaz)\ **Post date:** [July 21, 2016, 6:34am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/9 "2016-07-21T06:34:10Z")
Thanks Daniel, we are short in timeline, so for now I will upgrade to 2.3... ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [July 21, 2016, 6:58am UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/10 "2016-07-21T06:58:48Z")
Oh, sure. This makes sense then. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:33pm UTC](https://discuss.elastic.co/t/how-to-reindex-doc-with-multifield/55964/11 "2017-07-05T22:33:47Z")