# ES 2.1.1 poor indexing performance compared to 1.7.3

**URL:** https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491
**Category:** Elasticsearch
**Created:** [January 6, 2016, 10:46am UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491 "2016-01-06T10:46:08Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![zamblauskas](https://avatars.discourse-cdn.com/v4/letter/z/7cd45c/32.png) [@zamblauskas](https://discuss.elastic.co/u/zamblauskas)
#### Post date: [January 6, 2016, 10:46am UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/1 "2016-01-06T10:46:08Z")

</div>

Trying to understand the reason why 2.x ES has such poor indexing performance compared to the older 1.x.  
Vanilla ES install, simple index request:

- HDD machine: 1.7.3 ~20ms, 2.1.1 ~100ms.
- SSD machine: 1.7.3 ~20ms, 2.1.1 ~25ms.

Seems like related to some disk operation.  
Are there any configuration options we should consider to keep the performance similar to what it was with 1.x ES ?  
We are running single node, using it for real time application log monitoring (ELK stack), so doing mostly index operation.  
Any other suggestions would be appreciated.

`tar -xvf elasticsearch-1.7.3.tar.gz cd elasticsearch-1.7.3/ ./bin/elasticsearch time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY3QLS4y0FaHRm6Ax","_version":1,"created":true} real	0m1.377s user	0m0.000s sys	0m0.012s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY3tRS4y0FaHRm6Ay","_version":1,"created":true} real	0m0.019s user	0m0.007s sys	0m0.004s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY38tS4y0FaHRm6Az","_version":1,"created":true} real	0m0.027s user	0m0.004s sys	0m0.007s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY4KGS4y0FaHRm6A0","_version":1,"created":true} real	0m0.021s user	0m0.006s sys	0m0.005s`

`tar -xvf elasticsearch-2.1.1.tar.gz cd elasticsearch-2.1.1/ ./bin/elasticsearch time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZBOvQObEkTWKsmWJ","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real	0m1.226s user	0m0.012s sys	0m0.000s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZBtZQObEkTWKsmWK","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real	0m0.090s user	0m0.006s sys	0m0.006s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZB7uQObEkTWKsmWL","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real	0m0.098s user	0m0.009s sys	0m0.000s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZCK2QObEkTWKsmWM","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real	0m0.095s user	0m0.011s sys	0m0.000s`

For our production setup the performance downgrade is even more dramatic - 1.x ~2000 index requests / sec vs 2.x ~30 index requests / sec on HDD machine, but first would like to get help on understanding the poor performance of vanilla setup.

---

<div class="post-metadata">

### Author: ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [January 6, 2016, 12:46pm UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/2 "2016-01-06T12:46:56Z")

</div>

Elasticsearch 2.x has changed behavior for translog write sync. It fsyncs after every operation.

See [https://github.com/elastic/elasticsearch/issues/14399#issuecomment-152744759](https://github.com/elastic/elasticsearch/issues/14399#issuecomment-152744759)

Documentation is still saying that it's 5 seconds, but that is no longer the case.

The problem with that decision is that it puts a high burden of extra I/O on spindle disks where the journal file system may not be capable to keep up with the extra writes to perform. You can either reinspect your file system mount options to tune the journal write operations, or reset the translog sync interval to the old settings of 5 seconds to remedy the situation.

---

<div class="post-metadata">

### Author: ![zamblauskas](https://avatars.discourse-cdn.com/v4/letter/z/7cd45c/32.png) [@zamblauskas](https://discuss.elastic.co/u/zamblauskas)
#### Post date: [January 6, 2016, 2:01pm UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/3 "2016-01-06T14:01:25Z")

</div>

`index.translog.durability: async` did solve our performance issue.  
Thank you for your help.

---

<div class="post-metadata">

### Author: ![bleskes](https://avatars.discourse-cdn.com/v4/letter/b/71c47a/32.png) [@bleskes](https://discuss.elastic.co/u/bleskes)
#### Post date: [January 6, 2016, 8:03pm UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/4 "2016-01-06T20:03:22Z")

</div>

Double checking - are you using the bulk API to index? if so we only sync after every bulk request (if needed - i.e., if not already synced by another bulk). If not, moving the the bulk API will help greatly with indexing speed.

---

<div class="post-metadata">

### Author: ![zamblauskas](https://avatars.discourse-cdn.com/v4/letter/z/7cd45c/32.png) [@zamblauskas](https://discuss.elastic.co/u/zamblauskas)
#### Post date: [January 7, 2016, 9:56am UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/5 "2016-01-07T09:56:43Z")

</div>

No, we are not using the bulk API. We are satisfied with the performance (with `durability: async`) and using single request per index operation helps keep the code little bit simpler.  
But thanks for the tip, once our usage increases we'll look into switching to bulk operations.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 7, 2016, 12:54pm UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/6 "2016-01-07T12:54:32Z")

</div>

If you are looking for efficiency, I'd really recommend using bulk. It's really really much faster.

Bonus: you probably won't need to change `durability` settings which is risky IMO as you may loose data if unlucky...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:26pm UTC](https://discuss.elastic.co/t/es-2-1-1-poor-indexing-performance-compared-to-1-7-3/38491/7 "2017-07-05T23:26:24Z")

</div>


