# Performance of search requests in cluster from Kibana

**URL:** <https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997>\
**Category:** Elasticsearch\
**Created:** [November 24, 2017, 9:26am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997 "2017-11-24T09:26:37Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Evgeniy\_Ivlev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgeniy_ivlev/32/25860_2.png) [@Evgeniy\_Ivlev](https://discuss.elastic.co/u/Evgeniy_Ivlev)\
**Post date:** [November 24, 2017, 9:26am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/1 "2017-11-24T09:26:38Z")

</div>

Hi!

I deployed new ES cluster with following configuration:  
3 master nodes (4Gb RAM per node)  
2 data nodes (24Gb RAM per node), Xmx 12G  
1 node for Kibana (4 Gb RAM)

All nodes are virtual machines.  
lscpu shows following hardware info:  
CPU(s): 4  
Model name: Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz  
CPU MHz: 2599.998  
L1d cache: 32K  
L1i cache: 32K  
L2 cache: 256K  
L3 cache: 40960K

I'm going to use cluster for analytics system. I have following index configuratiuon:  
Every day 12 new indexes are created from ES template.  
The name of index is rt-geoID-YYYY-MM-DD  
number of shards: 1  
number of replicas: 1  
Each index has size 1-8 Gb

Data source is connected to DataNode2, Kibana is connected to DataNode1.  
There are no problems with data write and indexsing.  
But there are large problems with speed of search. Look at image below.  
When user performs search request from kibana by pattern rt-\* for sevaral days, only DataNode2 has huge % utulization of disk.  
I expect search request will be distributed between 2 data nodes.

 ![iostat](https://us1.discourse-cdn.com/elastic/original/3X/b/c/bc6f950f098b813caba71f435e72aacb1036685a.png)

Also I noticed If I search something by keyword field, first request is very long (Up to 90 sec), second one can be 2 times faster and more. Is there a way to cache something in order to perform first request fast?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 24, 2017, 9:38am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/2 "2017-11-24T09:38:22Z")

</div>

Which version of Elasticsearch are you using? This may be related to the issue discussed [in this thread](https://discuss.elastic.co/t/load-not-evenly-distributed/101343/9), which resulted in [this GitHub issue](https://github.com/elastic/elasticsearch/issues/24642). It looks like this should have been fixed in 6.0.

---

<div class="post-metadata">

**Author:** ![Evgeniy\_Ivlev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgeniy_ivlev/32/25860_2.png) [@Evgeniy\_Ivlev](https://discuss.elastic.co/u/Evgeniy_Ivlev)\
**Post date:** [November 24, 2017, 9:53am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/3 "2017-11-24T09:53:40Z")

</div>

5.6.4 version of ES and Kibana

---

<div class="post-metadata">

**Author:** ![Evgeniy\_Ivlev](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evgeniy_ivlev/32/25860_2.png) [@Evgeniy\_Ivlev](https://discuss.elastic.co/u/Evgeniy_Ivlev)\
**Post date:** [November 24, 2017, 11:26am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/4 "2017-11-24T11:26:21Z")

</div>

I'd like to understand - if it possible to serve single request by 2 nodes? Or in the best case different request may be served by different nodes?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 24, 2017, 11:31am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/5 "2017-11-24T11:31:18Z")

</div>

A single request can be served by shards on both nodes, leading to the query load being spread out. It seems, based on the GitHub issue I linked to, that Kibana adds preference, which under some circumstances can lead to happening properly.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 22, 2017, 11:31am UTC](https://discuss.elastic.co/t/performance-of-search-requests-in-cluster-from-kibana/108997/6 "2017-12-22T11:31:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
