# Problems with performance in ElasticSearch 2.3.2

**URL:** <https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035>\
**Category:** Elasticsearch\
**Created:** [May 3, 2016, 9:55am UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035 "2016-05-03T09:55:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guillermo\_Ortiz](https://avatars.discourse-cdn.com/v4/letter/g/258eb7/32.png) [@Guillermo\_Ortiz](https://discuss.elastic.co/u/Guillermo_Ortiz)\
**Post date:** [May 3, 2016, 9:55am UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/1 "2016-05-03T09:55:52Z")

</div>

I have migrated ES 2.2.1 to 2.3.2 and I'm having so problems with performance.  
I have one master node and five data nodes. Before I indexed 25K-30K/sec, now I index about 5K.  
I use Spark with es-hadoop to index the documents, I have tried with the 2.2.1 and 2.3.0 es-hadoop version with the same performance. It's the same code, same Spark version, same machines.

I checked as well all logs from ES nodes with any error.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 4, 2016, 4:53am UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/2 "2016-05-04T04:53:02Z")

</div>

Are you monitoring everything with Marvel to see what the cluster is doing?

---

<div class="post-metadata">

**Author:** ![Guillermo\_Ortiz](https://avatars.discourse-cdn.com/v4/letter/g/258eb7/32.png) [@Guillermo\_Ortiz](https://discuss.elastic.co/u/Guillermo_Ortiz)\
**Post date:** [May 4, 2016, 7:12am UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/3 "2016-05-04T07:12:59Z")

</div>

Yes, I have Marvel and Shield. I updated these plugins as well.  
The cluster is green and CPU, memory and IO are low, around 10-20%.

How I didn't get that cluster works well, I removed all indices and restart the cluster, but it fixed nothing.

I checked with nmon, top and so on all nodes and everything is right. Same that when I had ES 2.2.x

---

<div class="post-metadata">

**Author:** ![Jurg\_van\_Vliet](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jurg\_van\_Vliet](https://discuss.elastic.co/u/Jurg_van_Vliet)\
**Post date:** [May 16, 2016, 7:58am UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/4 "2016-05-16T07:58:08Z")

</div>

we are experiencing the same, starting when we moved to 2.3.1.

our production cluster has 1,254,024,220 docs, spread over 17 indexes with 600+ shards (2 replicas). we index nearly 30 docs per second. we have monthly logstash derived indexes. we have a total of 2.45TB of data in this cluster. we have 3 dedicated masters, 6 data nodes (16GB) and 2 client nodes.

performance of our own application is still ok, but kibana has a hard time. when the current index is 'cold' querying the cluster with kibana (with the default of latest 15 minutes) breaks the data. the cluster becomes unresponsive, indexing sometimes stops completely. after trying for a while kibana start working again. then it is relatively ok. (but we have to keep using it.)

CPU doesn't appear to be the bottleneck. the biggest problem is with the load. we see high io wait, higher than before. we run our data instances with RAID 0 of 16 disks of 160GB each. before moving to 2.3 we ran with 8 disks of 320GB.

we are in the process of reindexing older indexes, to make use of doc\_values. don't know if that is a good idea considering the load we see. and we are in the process of upgrading our staging cluster (similar setup, much less data) to 2.3.2, with marvel.

groet,  
jurg.

---

<div class="post-metadata">

**Author:** ![Jurg\_van\_Vliet](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jurg\_van\_Vliet](https://discuss.elastic.co/u/Jurg_van_Vliet)\
**Post date:** [May 19, 2016, 7:58pm UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/5 "2016-05-19T19:58:21Z")

</div>

looks like upgrading to kibana 4.5.1 did the trick. cluster stays healthy, kibana is consistently performant.

groet,  
jurg.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:50pm UTC](https://discuss.elastic.co/t/problems-with-performance-in-elasticsearch-2-3-2/49035/6 "2017-07-05T22:50:20Z")

</div>


