# Changing mapping

**URL:** <https://discuss.elastic.co/t/changing-mapping/94981>\
**Category:** Elasticsearch\
**Created:** [July 28, 2017, 3:02pm UTC](https://discuss.elastic.co/t/changing-mapping/94981 "2017-07-28T15:02:52Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![vinpereira](https://avatars.discourse-cdn.com/v4/letter/v/977dab/32.png) [@vinpereira](https://discuss.elastic.co/u/vinpereira)\
**Post date:** [July 28, 2017, 3:02pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/1 "2017-07-28T15:02:52Z")

</div>

Hi everyone,

Finally, my company will switch from ES 1.7 to ES 5.5 (in fact, the new one is for new projects ... and the old for that already exists ... anyway) !

> **"Old" Mapping**
>
> ```
> {
> "properties": {
> // some fields
> 
> "tasks": {
> "properties": {
> "formKey": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "assignee": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "endTime": {
> "type": "date"
> },
> "id": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "startTime": {
> "type": "date"
> },
> "duration": {
> "type": "long"
> },
> "category": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "description": {
> "type": "string"
> },
> "priority": {
> "type": "integer"
> },
> "name": {
> "type": "string"
> },
> "taskDefinitionKey": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "owner": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "dueDate": {
> "type": "date"
> },
> "deleteReason": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> }
> },
> "type": "nested"
> },
> "variables": {
> "properties": {
> "id": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "attachmentValue": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "storedType": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "longValue": {
> "type": "long"
> },
> "name": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "executionId": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "binaryValue": {
> "type": "binary"
> },
> "stringValue": {
> "type": "string"
> },
> "rawValue": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "doubleValue": {
> "type": "double"
> },
> "dateValue": {
> "type": "date"
> },
> "booleanValue": {
> "type": "boolean"
> },
> "originalType": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> }
> },
> "type": "nested"
> }
> }
> }
> 
> ```

> **"New" Mapping**
>
> ```
> {
> "properties": {
> // same fields until tasks
> 
> "tasks": { // What can I do with tasks? I thought to put them in another mapping, as parent/child... but I don't know if it would be good
> "properties": {
> "formKey": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "assignee": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "endTime": {
> "type": "date"
> },
> "id": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "startTime": {
> "type": "date"
> },
> "duration": {
> "type": "long"
> },
> "category": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "description": {
> "type": "string"
> },
> "priority": {
> "type": "integer"
> },
> "name": {
> "type": "string"
> },
> "taskDefinitionKey": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "owner": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> },
> "dueDate": {
> "type": "date"
> },
> "deleteReason": {
> "index": "not_analyzed",
> "doc_values": true,
> "type": "string"
> }
> },
> "type": "nested"
> },
> "variables": { // Now variables has a more simple structure. I will use dynamic_template here
> "properties": {
> "protocol": {
> "type": "keyword"
> },
> 
> // another fields, but with the same idea...
> 
> "eps": {
> "type": "keyword"
> }
> },
> "type": "nested" // I kept the nested type to keep them organized
> }
> }
> }
> 
> ```

About the old mapping:

- We have problems filtering the contents of variables and tasks  
Example: to get a protocol variable, we need to do a "[variables.name](http://variables.name) = protocol" AND "variables.stringValue = protocol\_value". If we try to use \__source include_ with "[variables.name](http://variables.name)", it will not work, because every "[variables.name](http://variables.name)" will return. The same idea for tasks.

- We have no relationship between "processA" and "subProcessA" in a case like "one processA has many subProcessA"

About the new mapping:

- Variables will be more direct. Something like "protocol = protocol\_value". This way, \__source include_ will work properly

- I will use parent/child for "processA" and "subProcessA" (both with a similar mapping)

- **But what can I do with the tasks?**  
The structure is fixed, it does not have much to take out or put in the fields. But the way it is, the same problem (which I explained with the variables) would continue. But if you do a type for tasks, that would be very repetitive. I would have processA tasks and subProcessA tasks. Just thinking about these cases. I would still have processB, subProcessB and tasks for both ... and there are more process yet.

Do you have any suggestion?

thanks

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [August 4, 2017, 10:37am UTC](https://discuss.elastic.co/t/changing-mapping/94981/2 "2017-08-04T10:37:51Z")

</div>

Hello Vinicius,

Before we go into the task question, I would like to first discuss a little bit further you `protocol` structure change. What goes into `protocol`? Is this some sort of `id` that can grow unbounded or is it a known, fixed, list of values?

Cheers

---

<div class="post-metadata">

**Author:** ![vinpereira](https://avatars.discourse-cdn.com/v4/letter/v/977dab/32.png) [@vinpereira](https://discuss.elastic.co/u/vinpereira)\
**Post date:** [August 9, 2017, 2:58pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/3 "2017-08-09T14:58:08Z")

</div>

protocol is an ID, but it is one of the possible variables. Others are "eps" (a subarea for a company), some booleans (for BPM control), some strings with internal meaning for the area...

a little more about procotol: it has a different format for each macro-area, for example telecom is a string 'XXXXXXX-YEAR', with N Xs.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [August 10, 2017, 12:34am UTC](https://discuss.elastic.co/t/changing-mapping/94981/4 "2017-08-10T00:34:43Z")

</div>

So you should avoid using `protocol = protocol_value` otherwise every new protocol id will be a mapped field and you will have serious performance issues.

I recommend that you keep the old way and use nested objects or parent/child

---

<div class="post-metadata">

**Author:** ![vinpereira](https://avatars.discourse-cdn.com/v4/letter/v/977dab/32.png) [@vinpereira](https://discuss.elastic.co/u/vinpereira)\
**Post date:** [August 10, 2017, 1:34pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/5 "2017-08-10T13:34:02Z")

</div>

But it's only one protocol per document... it's a id, but it's not the document id (this is for the process id).

Here is a example of the problem.  
First a return with the old (current) mapping:

- Query  
 ![Screenshot from 2017-08-10 10-29-22](https://us1.discourse-cdn.com/elastic/original/3X/f/5/f5662f045405bae8522e98d2a53a4bf13ecb7a7c.png)

- Result  

It's not possible to filter only the "protocol", because it's not a field (it's a value)

Now the new mapping:

- Query  
 ![Screenshot from 2017-08-10 10-30-21](https://us1.discourse-cdn.com/elastic/original/3X/0/b/0bac99142926a8b71713b79544422b26d23ae3a9.png)

It's easier to filter/query

- Result  
 ![Screenshot from 2017-08-10 10-30-34](https://us1.discourse-cdn.com/elastic/original/3X/f/8/f87895ee2f81293f7d11f1a012828f8f3e16db93.png)

And the result only has the "variables.protocol" field.

I'm using dynamic mapping and dynamic template to do this.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [August 10, 2017, 1:38pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/6 "2017-08-10T13:38:38Z")

</div>

Ok, now I understand it better. I was with a different understanding, sorry.

How many types of variables exists in total?

---

<div class="post-metadata">

**Author:** ![vinpereira](https://avatars.discourse-cdn.com/v4/letter/v/977dab/32.png) [@vinpereira](https://discuss.elastic.co/u/vinpereira)\
**Post date:** [August 10, 2017, 2:31pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/7 "2017-08-10T14:31:17Z")

</div>

N types. Depends of the project/area.

That is why I'm thinking about dynamic template...

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [August 10, 2017, 3:05pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/8 "2017-08-10T15:05:16Z")

</div>

If there is an unbounded number of variable types, then the old way is better. Otherwise, every time you index a document with different variables, new fields will be created and the bigger the mapping, the worse.

For instance, if you index one document like:

```json
{
  "variables": {
      "protocolo": "2106087-2017"
  }
}

```

And another one line

```json
{
  "variables": {
      "another_var": "another_value"
  }
}

```

Then this document's mapping will have created both "protocolo" field and "another\_var" field and this is a mapping that grows unbounded, potentially. When that mapping hits +500 fields then cluster performance will degrade and you will need to scale up the servers (instead of scale out) to effectively solve the issue.

Bottom line: if you don't know in advance that the number of variable types can stay low (like 50-100 variable types), then you should stick with old mapping and use nested object or parent/child.

---

<div class="post-metadata">

**Author:** ![vinpereira](https://avatars.discourse-cdn.com/v4/letter/v/977dab/32.png) [@vinpereira](https://discuss.elastic.co/u/vinpereira)\
**Post date:** [August 14, 2017, 6:35pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/9 "2017-08-14T18:35:58Z")

</div>

Thanks Thiago!

I just used the 'protocol' as a universal example.  
For default, some variables are know (most of them in a project), like protocol, eps, shouldCreate, etc. They will be present in the mapping (not dynamic).

I want to use dynamic mapping and template for new variables, so the developer does not need to first change the mapping and then run the process with the new variable. I want to automate this.

Right now, the project with more variables has 86 variables. All them non-dynamic.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [August 14, 2017, 6:49pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/10 "2017-08-14T18:49:24Z")

</div>

So if you keep the variables as low as 86 types, then it's all good and dynamic templates is the way to go 😉

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 11, 2017, 6:49pm UTC](https://discuss.elastic.co/t/changing-mapping/94981/11 "2017-09-11T18:49:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.