# Very Slow incrementing counter

**URL:** <https://discuss.elastic.co/t/very-slow-incrementing-counter/55114>\
**Category:** Elasticsearch\
**Created:** [July 9, 2016, 2:41pm UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114 "2016-07-09T14:41:27Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![loske](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@loske](https://discuss.elastic.co/u/loske)\
**Post date:** [July 9, 2016, 2:41pm UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/1 "2016-07-09T14:41:27Z")

</div>

Hello,  
I have a counter which i need to increment :

`curl -XPOST 'localhost:9200/something/ble/372204/_update' -d' { "script": { "inline": "ctx._source.counter++" } }'`

This index has around ~4k elements but every counter increment takes minimum 160-300ms. Incrementing the same counter in postgres on same data also with index takes 0.01 ms. I understand that one is written in C and other in Java but both make reindex and elastic is not 10-20x slower it's 1600x times slower.

Elastic version 2.3.4  
Config/init changes:

ES\_HEAP\_SIZE=3g  
bootstrap.mlockall: true  
LimitMEMLOCK=infinity

Stats are here:

[http://pastebin.com/fW9nS0Wi](http://pastebin.com/fW9nS0Wi)

---

<div class="post-metadata">

**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\
**Post date:** [July 9, 2016, 10:42pm UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/2 "2016-07-09T22:42:44Z")

</div>

Incrementing the counter requires reindexing the whole document. In a  
system like postgresql that just involves copying the row, probably to the  
same block. Usually you don't even need to mess with indexes. Elasticsearch  
doesn't work like that. It has to reanalyze all the fields and eventually  
build a new segment, inverted index, doc values, and all.

Relational databases make tradeoffs to make updates faster. Elasticsearch  
makes tradeoffs to make aggregations and full text search faster. Those  
tradeoffs are baked in at a fairly deep level.

That said 160ms is quite a bit. You have lots of fields in that document?  
Network between the shards slowish? I dunno, hard to say.

---

<div class="post-metadata">

**Author:** ![loske](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@loske](https://discuss.elastic.co/u/loske)\
**Post date:** [July 10, 2016, 1:33am UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/3 "2016-07-10T01:33:06Z")

</div>

Hi Nik,  
Thx for replying. I don't have a lof of fields i think around 35. It's not network, it's on one server and i am doing local requests, time is spent inside elasticsearch. Any way to check why it's so slow?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 10, 2016, 5:49am UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/4 "2016-07-10T05:49:50Z")

</div>

Can you try with an indexed script?

[https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-scripting.html#\_indexed\_scripts](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-scripting.html#_indexed_scripts)

Are you using SSD drives?

---

<div class="post-metadata">

**Author:** ![loske](https://avatars.discourse-cdn.com/v4/letter/l/bc79bd/32.png) [@loske](https://discuss.elastic.co/u/loske)\
**Post date:** [July 10, 2016, 4:23pm UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/5 "2016-07-10T16:23:53Z")

</div>

Hi it dosen't change anything. I am not using SSD.

Any way to profile elastic and see where it's spending it's time on this query ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 11, 2016, 8:51am UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/6 "2016-07-11T08:51:03Z")

</div>

Using SSD could help a bit here. Also from 5.0 switching to [Painless scripting language](https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-painless.html#modules-scripting-painless) could help a bit.

You could run `hot_threads` API while the query is running and see what is happening.

I don't remember exactly the code but here are the main steps elasticsearch has to run when you call this script:

- compile the script (might be cached IIRC)
- load the document
- execute the script
- delete the old version of the document
- put the new version of the document (which leads to all what @nik9000 [described](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/2?u=dadoonet))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:36pm UTC](https://discuss.elastic.co/t/very-slow-incrementing-counter/55114/7 "2017-07-05T22:36:39Z")

</div>


