# Bulk update taking time on updating nested properties

**URL:** <https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810>\
**Category:** Elasticsearch\
**Created:** [July 2, 2015, 2:08pm UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810 "2015-07-02T14:08:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sandeep\_Verma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeep_verma/32/12517_2.png) [@Sandeep\_Verma](https://discuss.elastic.co/u/Sandeep_Verma)\
**Post date:** [July 2, 2015, 2:08pm UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/1 "2015-07-02T14:08:39Z")

</div>

I am running bulk query on elastic search for updating user's nested properties.

Below is the query which I am executing to update user's properties. It updates around 200 properties of single user.  
It takes around 5000 mili seconds

Machine configuration on Amazon Aws  
Master : 1, c4.large  
Data Nodes : 2 c4.2xlarge

POST /INDEX\_NAME/user/\_bulk  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) { if(item['name']==null){item['name'] =name;} } }","params":{"key":"26069\_purchased","name":"26069"}}  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) { if(item['action']==null){item['action'] =action;} } }","params":{"key":"26069\_purchased","action":"purchased"}}  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) {if(item['price']==null){item['price']=price;}else{ item['price']=item['price']+ price;} } }","params":{"key":"26069\_purchased","price":0}}  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) {if(item['count']==null){item['count']=count;}else{ item['count']=item['count']+ count;} } }","params":{"key":"26069\_purchased","count":1}}  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) {if(item['last\_date']==null){item['last\_date']=last\_date;}else{ item['last\_date'] = last\_date; } }}","params":{"key":"26069\_purchased","last\_date":"2015-06-20T15:37:09"}}  
{"update":{"\_id":"55950ed51d4213241f7c38b4","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5,"refresh":true}}  
{"lang":"mvel","script":"foreach (item : ctx.\_source.category) { if (item['key'] == key) {if(item['without\_promo']==null){item['without\_promo']=without\_promo;}else{ item['without\_promo']=item['without\_promo']+ without\_promo;} } }","params":{"key":"26069\_purchased","without\_promo":1}}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 2, 2015, 11:44pm UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/2 "2015-07-02T23:44:31Z")

</div>

When you update a nested doc it needs to do a delete of the original and then reindex the new one.  
Given you are doing this for a single doc that's a lot of delete+index it needs to do.

Why not just roll these into a single update?

---

<div class="post-metadata">

**Author:** ![Sandeep\_Verma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeep_verma/32/12517_2.png) [@Sandeep\_Verma](https://discuss.elastic.co/u/Sandeep_Verma)\
**Post date:** [July 4, 2015, 7:16am UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/3 "2015-07-04T07:16:46Z")

</div>

i will change my update in to single update by doing this.But there is no change. it still taking more than 3000 ms .

{"update":{"\_id":"hr324hdie93hdjdo391","type":"user","index":"jal","routing":2,"\_retry\_on\_conflict":5}}  
{"lang":"mvel","script":"if(ctx.\_source.containsKey("category")){i=0; io=0; foreach (item : ctx.\_source.category){i++; if (item['key'] == v\_123\_purchased){ io=1}} if(io==0){ctx.\_source.category.add(p\_123\_purchased);}}else{ctx.\_source.category=c\_123\_purchased;}if(ctx.\_source.containsKey("category")){i=0; io=0; foreach (item : ctx.\_source.category){i++; if (item['key'] == v\_233\_purchased){ io=1}} if(io==0){ctx.\_source.category.add(p\_233\_purchased);}}else{ctx.\_source.category=c\_233\_purchased;}foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) { if(item['name']==null){item['name'] =name\_123\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) { if(item['action']==null){item['action'] =action\_123\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) {if(item['price']==null){item['price']=price\_123\_purchased;}else{ item['price']=item['price']+ price\_123\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) {if(item['count']==null){item['count']=count\_123\_purchased;}else{ item['count']=item['count']+ count\_123\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) {if(item['last\_date']==null){item['last\_date']=last\_date\_123\_purchased;}else{ item['last\_date'] = last\_date\_123\_purchased; } }}foreach (item : ctx.\_source.category) { if (item['key'] == key\_123\_purchased) {if(item['with\_promo']==null){item['with\_promo']=with\_promo\_123\_purchased;}else{ item['with\_promo']=item['with\_promo']+ with\_promo\_123\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) { if(item['name']==null){item['name'] =name\_233\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) { if(item['action']==null){item['action'] =action\_233\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) {if(item['price']==null){item['price']=price\_233\_purchased;}else{ item['price']=item['price']+ price\_233\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) {if(item['count']==null){item['count']=count\_233\_purchased;}else{ item['count']=item['count']+ count\_233\_purchased;} } }foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) {if(item['last\_date']==null){item['last\_date']=last\_date\_233\_purchased;}else{ item['last\_date'] = last\_date\_233\_purchased; } }}foreach (item : ctx.\_source.category) { if (item['key'] == key\_233\_purchased) {if(item['with\_promo']==null){item['with\_promo']=with\_promo\_233\_purchased;}else{ item['with\_promo']=item['with\_promo']+ with\_promo\_233\_purchased;} } }","params":{"p\_123\_purchased":{"key":"123\_purchased"},"c\_123\_purchased":[{"key":"123\_purchased"}],"v\_123\_purchased":"123\_purchased","p\_233\_purchased":{"key":"233\_purchased"},"c\_233\_purchased":[{"key":"233\_purchased"}],"v\_233\_purchased":"233\_purchased","key\_123\_purchased":"123\_purchased","name\_123\_purchased":123,"action\_123\_purchased":"purchased","price\_123\_purchased":20,"count\_123\_purchased":1,"last\_date\_123\_purchased":"2015-07-04T07:11:03","with\_promo\_123\_purchased":1,"key\_233\_purchased":"233\_purchased","name\_233\_purchased":233,"action\_233\_purchased":"purchased","price\_233\_purchased":20,"count\_233\_purchased":1,"last\_date\_233\_purchased":"2015-07-04T07:11:03","with\_promo\_233\_purchased":1}}

Here is my Node status

 ![](https://us1.discourse-cdn.com/elastic/original/2X/0/0c527415ca9cdbfd2e8b775a454777bd06e385b7.png)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 4, 2015, 7:41am UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/4 "2015-07-04T07:41:18Z")

</div>

Then the problem is likely to be your heavy use of scripting, not sure what else can be done here.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 4, 2015, 8:06am UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/5 "2015-07-04T08:06:11Z")

</div>

You should IMO create a script in config dir and use it or at least use parameters because I think you script have to be compiled for each single line you send which is an heavy operation.

---

<div class="post-metadata">

**Author:** ![Sandeep\_Verma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sandeep_verma/32/12517_2.png) [@Sandeep\_Verma](https://discuss.elastic.co/u/Sandeep_Verma)\
**Post date:** [July 4, 2015, 10:23am UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/6 "2015-07-04T10:23:59Z")

</div>

Hii david can you please give me any example that how to create script in config file. Or any simple method to update nested object . I just want to update property of nested obj by four ways append , increment , ignore ,replace with new value .  
My nested Obj lokking like this

```
{
               "_userId": "hr324hdie93hdjdo391",
               "_shardId": 2,
               "brand": [
                  {
                     "price": 140,
                     "name": 23,
                     "count": 7,
                     "without_promo": 7,
                     "action": "purchased",
                     "key": "23_purchased",
                     "last_date": "2015-07-04T10:10:25"
                  }
               ],
               "product": [
                  {
                     "price": 140,
                     "with_promo": 7,
                     "name": 28,
                     "count": 7,
                     "action": "purchased",
                     "key": "28_purchased",
                     "last_date": "2015-07-04T10:10:25"
                  },
                  {
                     "price": 40,
                     "with_promo": 2,
                     "name": 29,
                     "count": 2,
                     "action": "purchased",
                     "key": "29_purchased",
                     "last_date": "2015-07-04T06:25:30"
                  }
               ],
               "category": [
                  {
                     "price": 140,
                     "with_promo": 7,
                     "name": 123,
                     "count": 7,
                     "action": "purchased",
                     "key": "123_purchased",
                     "last_date": "2015-07-04T10:10:25"
                  },
                  {
                     "price": 40,
                     "with_promo": 2,
                     "name": 124,
                     "count": 2,
                     "action": "purchased",
                     "key": "124_purchased",
                     "last_date": "2015-07-04T06:25:30"
                  }
               ]
            }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:03am UTC](https://discuss.elastic.co/t/bulk-update-taking-time-on-updating-nested-properties/24810/7 "2017-07-06T00:03:35Z")

</div>