# Uneven search load on data nodes

**URL:** <https://discuss.elastic.co/t/uneven-search-load-on-data-nodes/19870>\
**Category:** Elasticsearch\
**Created:** [September 18, 2014, 4:50pm UTC](https://discuss.elastic.co/t/uneven-search-load-on-data-nodes/19870 "2014-09-18T16:50:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manish\_Garg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manish_garg/32/1274_2.png) [@Manish\_Garg](https://discuss.elastic.co/u/Manish_Garg)\
**Post date:** [September 18, 2014, 4:50pm UTC](https://discuss.elastic.co/t/uneven-search-load-on-data-nodes/19870/1 "2014-09-18T16:50:02Z")

</div>

We have built elastic search cluster for our application autosuggest and we  
are facing one strange issue, need immediate help. Below is the setup of ES  
cluster:

We built the azure elastic search cluster with:

Master only Nodes: 3 medium VMs with 7 GB RAM

Non-master non-data nodes: 3 medium VMs with 7 GB RAM

Data nodes: 3 large VMs with 14 GB RAM

ES version: 1.1.1

Java version: 1.7.0\_45

Index size is: 1GB

Replica:1

Shards: 5

Java Heap size is half of RAM size.

In a definite pattern everyday, we see the number of requests increases  
x100. I think our cluster should be able to handle that load, but 99% of  
search requests goes only on one node(and everyday that node keeps  
changing), and other node have only 1% traffic. Because of this, the search  
thread pool queue increases on that node. If that load would have divided  
on all the data nodes then we wouldn't have had any problem. I have  
increased the search thread pool count to 2000 on each of data nodes. But  
when requests increases on one node all threads get occupied and queue  
start building. Today I mitigated it by setting the search thread pool type  
to cached (that will have unbounded search thread pool).

Please let me know if anyone faced similar issue or know how to make the  
load balancer work properly. We are using bigdesk to monitor the traffic,  
please let me know if you need any more information.

Thanks,

Manish

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAO4PsGGJkP5pwCypegbnS\_ik3%3D8NVMe3b06f\_3gZgQ%3DQHrpXag%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAO4PsGGJkP5pwCypegbnS_ik3%3D8NVMe3b06f_3gZgQ%3DQHrpXag%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:01am UTC](https://discuss.elastic.co/t/uneven-search-load-on-data-nodes/19870/2 "2017-07-06T01:01:13Z")

</div>