# Multiple update of one document in one \_bulk too slow

**URL:** <https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885>\
**Category:** Elasticsearch\
**Created:** [April 19, 2017, 12:26pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885 "2017-04-19T12:26:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dizzzyroma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dizzzyroma/32/64191_2.png) [@dizzzyroma](https://discuss.elastic.co/u/dizzzyroma)\
**Post date:** [April 19, 2017, 12:26pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/1 "2017-04-19T12:26:40Z")

</div>

Hi,

Multiple update of one document in one \_bulk is too slow

Example:

```
curl -XPOST "http://localhost:9200/_bulk" -H 'Content-Type: application/json' -d'
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":1}}
{"update":{"_index":"cc_36","_type":"job","_id":"960_87786b7c-69df-4d3d-b90b-b9b26f08e2a1"}}
{"doc":{"status":2}}
'

```

Elasticsearch version: 5.3, 5.2

Plugins installed:

JVM version: 1.8.0\_121-b13

OS version:Ubuntu 16.04.2 LTS

bulk request with 75 updates takes near 6 seconds on our test index that runs on c3.xlarge (4 CPU, 8 GB, documents in inex 14860619, index size 20GB) .  
bulk request with 150 updates takes near 12 seconds.

bulk index with 75 documents takes 0.2 seconds

On ES 2.4 - we haven't this problem

Profiler info:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/8/f/8f1e7142add611ef18f781e71cb29d57d01f1bf1.png)  
 ![](https://us1.discourse-cdn.com/elastic/original/3X/8/3/83ba70ee30d2ef3b4c0f4a60f57a924e3bd2a7de.png)

If it helps, I may open profiling port to cluster where this problem is reproduced

I create bug on this [https://github.com/elastic/elasticsearch/issues/24172](https://github.com/elastic/elasticsearch/issues/24172) , where advised me to turn off global ordinals.

I turn off global ordinals in mapping for child and reindex all. But it not helps ☹  
Also I am set refresh\_interval to -1. The same result.

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [April 19, 2017, 12:53pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/2 "2017-04-19T12:53:28Z")

</div>

Just linking my last comment on the issue:

> <https://github.com/elastic/elasticsearch/issues/24172#issuecomment-295232576>

I think the best for you at this point is to rethink how you send documents to elasticsearch.  
You should try to lower the number of updates for a single document, you can do it per batch for instance.  
Though turning off global ordinals loading on refresh should have helped. Do you still see buildOrdinalMap in your profiler when you disable this option ?

---

<div class="post-metadata">

**Author:** ![dizzzyroma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dizzzyroma/32/64191_2.png) [@dizzzyroma](https://discuss.elastic.co/u/dizzzyroma)\
**Post date:** [April 19, 2017, 1:05pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/3 "2017-04-19T13:05:18Z")

</div>

Yes

 ![](https://us1.discourse-cdn.com/elastic/original/3X/c/f/cfa58e8a2962ddf20ba5ca79c4e3bfe89a1ce549.png)  
 ![](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d4d6a3f488b3117267ac781980d3995b105e1c5c.png)

mapping

```
  "stat": {
    "dynamic": "false",
    "_all": {
      "enabled": false
    },
    "_parent": {
      "type": "job",
      "eager_global_ordinals": false
    },
    "_routing": {
      "required": true
    },
    "date_detection": false,
    "properties": {}}

```

Child documents in index is zero.  
I try to reproduce it without childs in mapping

---

<div class="post-metadata">

**Author:** ![dizzzyroma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dizzzyroma/32/64191_2.png) [@dizzzyroma](https://discuss.elastic.co/u/dizzzyroma)\
**Post date:** [April 19, 2017, 2:30pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/4 "2017-04-19T14:30:01Z")

</div>

Without child relation bulk request with 75 updates takes near 0.5 sec. More than two times slower than index 75 new documents

---

<div class="post-metadata">

**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\
**Post date:** [April 19, 2017, 2:45pm UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/5 "2017-04-19T14:45:43Z")

</div>

Yes this is expected if you update the same document multiple times in a short period. As I said in 5.x we changed the way we retrieve the documents for an update and if the documents has been updated shortly before the new update this will trigger an automatic refresh of your index. If you have a single document that you update multiple times in your bulk request it will be slower than in 2.x and there is nothing we can do about it. You should find another way to update this document, maybe make a single update after a certain amount of time ?

---

<div class="post-metadata">

**Author:** ![dizzzyroma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dizzzyroma/32/64191_2.png) [@dizzzyroma](https://discuss.elastic.co/u/dizzzyroma)\
**Post date:** [April 21, 2017, 11:00am UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/6 "2017-04-21T11:00:25Z")

</div>

Thanks for answer. Now we use two updates in one bulk for next situation:  
Example:  
we have doc in ES:

> "\_source": {  
> "id": 21129546,  
> "additional\_fields": {  
> "xxxx": "yyyy",  
> "zzz": "zzz"  
> }}

And we need update additional\_fields to {"aaa": "vvvvv"}

first update - additional\_fields: null  
second update additional\_fields: {"aaa": "vvvvv"}

Does ES have any mechanism to do it on one request?

I do it using scripting

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 19, 2017, 11:05am UTC](https://discuss.elastic.co/t/multiple-update-of-one-document-in-one--bulk-too-slow/82885/7 "2017-05-19T11:05:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.