Trying to understand the reason why 2.x ES has such poor indexing performance compared to the older 1.x.
Vanilla ES install, simple index request:
- HDD machine: 1.7.3 ~20ms, 2.1.1 ~100ms.
- SSD machine: 1.7.3 ~20ms, 2.1.1 ~25ms.
Seems like related to some disk operation.
Are there any configuration options we should consider to keep the performance similar to what it was with 1.x ES ?
We are running single node, using it for real time application log monitoring (ELK stack), so doing mostly index operation.
Any other suggestions would be appreciated.
tar -xvf elasticsearch-1.7.3.tar.gz cd elasticsearch-1.7.3/ ./bin/elasticsearch time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY3QLS4y0FaHRm6Ax","_version":1,"created":true} real 0m1.377s user 0m0.000s sys 0m0.012s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY3tRS4y0FaHRm6Ay","_version":1,"created":true} real 0m0.019s user 0m0.007s sys 0m0.004s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY38tS4y0FaHRm6Az","_version":1,"created":true} real 0m0.027s user 0m0.004s sys 0m0.007s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWY4KGS4y0FaHRm6A0","_version":1,"created":true} real 0m0.021s user 0m0.006s sys 0m0.005s
tar -xvf elasticsearch-2.1.1.tar.gz cd elasticsearch-2.1.1/ ./bin/elasticsearch time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZBOvQObEkTWKsmWJ","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real 0m1.226s user 0m0.012s sys 0m0.000s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZBtZQObEkTWKsmWK","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real 0m0.090s user 0m0.006s sys 0m0.006s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZB7uQObEkTWKsmWL","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real 0m0.098s user 0m0.009s sys 0m0.000s time curl -XPOST 'http://localhost:9200/a/a' -d '{}' {"_index":"a","_type":"a","_id":"AVIWZCK2QObEkTWKsmWM","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true} real 0m0.095s user 0m0.011s sys 0m0.000s
For our production setup the performance downgrade is even more dramatic - 1.x ~2000 index requests / sec vs 2.x ~30 index requests / sec on HDD machine, but first would like to get help on understanding the poor performance of vanilla setup.