# How to test application with multiple mapping types after their deprecation?
**URL:** https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740
**Category:** Elasticsearch
**Created:** [January 2, 2018, 8:20am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740 "2018-01-02T08:20:19Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)
#### Post date: [January 2, 2018, 8:20am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/1 "2018-01-02T08:20:19Z")
Hi,
We have some ruby scripts that aggregate and re-index data in ES. Whenever we make changes to these scripts we first run them locally to make sure we didn't break anything and that data is correctly indexed. I sometimes would remove my dev indices to keep them clean and aid my testing.
I totally didn't think about the fact that mapping types are now deprecated and removed my dev indices. Now I'm unable to run the scripts for testing purposes as I get "Rejecting mapping update to [dev] as the final mapping would have more than 1 type: [journal, account, order]".
Is there anything I can do to replicate the mapping we still have in production?
---
### Author:  [@Julien](https://discuss.elastic.co/u/Julien)
#### Post date: [January 2, 2018, 10:58am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/2 "2018-01-02T10:58:44Z")
Is prod and are the indices on version 5 ? If so, please look at the snapshot API here (I have not tested this but restoring snapshot of version 5 is supported for version 6):
[https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html)
Note that if you use version 6, you should reindex to new indices with a single type as any index created in version 6 cannot have multiple types, a few ways to do this are documented here:
[https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html)
---
### Author:  [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)
#### Post date: [January 2, 2018, 11:16am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/3 "2018-01-02T11:16:51Z")
🤦♀️ I thought I read somewhere that multiple mapping types created prior to 6 would still work but I forgot that every month we create new indices which with our prod ES upgraded to version 6 will break anyway. I better get reindexing then. Thanks for help. 🙂
---
### Author:  [@a5a](https://discuss.elastic.co/u/a5a)
#### Post date: [January 3, 2018, 3:25am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/4 "2018-01-03T03:25:30Z")
@Kornelia_Watson
[https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html#removal-of-types](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/removal-of-types.html#removal-of-types) If you click here, you'll see the sentence that explains exactly what you said: that multiple mapping types in indices created \<6 work.
Yeah, that makes sense, your newly created indices with your template will fail after you upgrade to 6. I'm working on a tool that will automate reindexing indices with multiple types into single-type indices, but until then, you can watch this webinar where I describe two strategies for solving the problem: [https://www.elastic.co/webinars/upgrading-your-elastic-stack](https://www.elastic.co/webinars/upgrading-your-elastic-stack), and this article which explains a third strategy: [https://www.elastic.co/blog/kibana-6-removal-of-mapping-types](https://www.elastic.co/blog/kibana-6-removal-of-mapping-types)
---
### Author:  [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)
#### Post date: [January 6, 2018, 11:35am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/5 "2018-01-06T11:35:36Z")
Thanks for the info. I've come across another problem when re-indexing and I didn't see it covered anywhere. I googled for some solutions but they don't quite cover my use case or can't get them to work.
Our mapping has changed over the last few months, for example, we used to have "order\_value\_USD" field but we then changed it to "value\_USD" and so some documents have "order\_value\_USD" and some "value\_USD". When trying to re-index documents I'm getting "dynamic introduction of [order\_value\_USD] within [order] is not allowed".
What is the best and most time efficient way to re-index old data in a way that it matches new mapping?
Also, when trying to re-index I keep getting 504s - "Gateway Time-out".
---
### Author:  [@a5a](https://discuss.elastic.co/u/a5a)
#### Post date: [January 7, 2018, 9:34pm UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/6 "2018-01-07T21:34:29Z")
You'll have to reindex with a script that transforms the old `order_value_USD` values into `value_USD` values.
```auto
ctx._source.value_USD = ctx._source.order_value_USD
```
I _think_ the above assumes that you have `order_value_USD` defined for all documents that the reindex runs on, so it will only work on that subset of documents. I can get more details during the work week.
There's a short section mentioning painless scripting in [Reindex API docs](https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-reindex.html) which can give you an example of how it can work. The [Removal of Mapping Types](https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html) doc also has examples of the script. Basically you'll need to tailor it to your situation.
---
### Author:  [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)
#### Post date: [January 9, 2018, 10:55am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/7 "2018-01-09T10:55:56Z")
Hi, so I tried reindexing query with your suggested script as well as with adding the "remove" bit, i.e.
> "ctx.\_source.value\_USD = ctx.\_source.remove('order\_value\_USD'); "
like suggested in [Reindex API | Elasticsearch Guide [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-reindex.html#docs-reindex-change-name)
So my overall query was:
> POST \_reindex
> {
> "conflicts": "proceed",
> "source": {
> "index": "insights\_prod-2017.07",
> "type": "order"
> },
> "dest": {
> "index": "orders\_prod-2017.07",
> "op\_type": "create"
> },
> "script": {
> "lang": "painless",
> "source":
> "ctx.\_source.value\_USD = ctx.\_source.remove('order\_value\_USD');
> ctx.\_source.value = ctx.\_source.remove('order\_value');
> ctx.\_source.net\_value\_excl\_shipping\_USD = ctx.\_source.remove('order\_net\_value\_excl\_shipping\_USD');
> ctx.\_source.net\_value\_excl\_shipping = ctx.\_source.remove('order\_net\_value\_excl\_shipping');
> ctx.\_source.invoiced\_timestamp = ctx.\_source.remove('order\_invoiced\_timestamp');
> ctx.\_source.fully\_shipped\_timestamp = ctx.\_source.remove('order\_fully\_shipped\_timestamp');
> ctx.\_source.remove('account\_is\_subscriber');
> ctx.\_source.remove('item\_count\_stock\_tracked');
> ctx.\_source.id = ctx.\_source.remove('order\_id');
> ctx.\_source.contact\_id = ctx.\_source.remove('contact\_email');"
> }
> }
So, all the missing documents got created, but all the values mentioned above are null even though they exist in original docs.
Any ideas what I'm doing wrong?
Also, any ideas what the deal is with
> {
> "statusCode": 504,
> "error": "Gateway Time-out",
> "message": "Client request timeout"
> }
I tried adding
> ?wait\_for\_completion=true
but it didn't change anything
---
### Author:  [@Kornelia\_Watson](https://discuss.elastic.co/u/Kornelia_Watson)
#### Post date: [January 10, 2018, 8:05am UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/8 "2018-01-10T08:05:46Z")
I managed to avoid a timeout by filtering which accounts to reindex.
> "query": {
> "bool": {
> "must": [
> {
> "query\_string": {
> "query": "account\_id: a\*",
> "analyze\_wildcard": true
> }
> }
> ]
> }
> }
I'll need to go through alphabet but I can cope with that 😛 Also, I removed
"conflicts": "proceed" and "op\_type": "create" as I was getting version conflicts.
Still no sign of missing values though 😕
---
### Author:  [@jimczi](https://discuss.elastic.co/u/jimczi)
#### Post date: [January 11, 2018, 2:50pm UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/9 "2018-01-11T14:50:06Z")
For the timeouts you should set `wait_for_completion=false` in your reindex query and monitor the execution of your tasks through the task API:
[https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-task-api](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-task-api)
If your reindex requests take some time to finish then the client connection can fail because it didn't receive any data before completion. This is why you need to perform this asynchronously, send the reindex request to es and then monitor the completion with another query.
Regarding the null values, can you try with a single document (check the original source and the new one) to see if your script works. And if it's not you can paste the full recreation here (the original document + the reindex request).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 8, 2018, 2:50pm UTC](https://discuss.elastic.co/t/how-to-test-application-with-multiple-mapping-types-after-their-deprecation/113740/10 "2018-02-08T14:50:27Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.