# Terribly slow kibana query

**URL:** https://discuss.elastic.co/t/terribly-slow-kibana-query/189010
**Category:** Kibana
**Created:** [July 5, 2019, 6:10am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010 "2019-07-05T06:10:47Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![mrashid](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mrashid](https://discuss.elastic.co/u/mrashid)
#### Post date: [July 5, 2019, 6:10am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/1 "2019-07-05T06:10:47Z")

</div>

I have around 2GB of events getting accumulated every day. I use curator to get rid of data every 5 days.

Now, the problem is it takes long time to execute search in kibana. I am using docker to run ELK . When I do docker stats , I can see that only 3 GB of memory is getting used when I have 64GB available .

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/7/1782b5d432d33c5ed91d065da8236e8d59b7e047.png)

I check the heap : It shows the max heap is 16GB (which according to [ergonomics algorithm is correct](http://www.oracle.com/technetwork/java/ergo5-140223.html).

According to the algorithm , The max heap is 1/4 of the total and the initial heap is 1/64 of the total . This is what i can see :

```
 java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|P ermSize|ThreadStackSize'
 intx CompilerThreadStackSize = 0 {pd product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx HeapSizePerGCThread = 87241520 {product}
uintx InitialHeapSize := 1054867456 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 16848519168 {product}
 intx ThreadStackSize = 1024 {pd product}
 intx VMThreadStackSize = 1024 {pd product}

```

**I have increased the shards to 15 and replicas to 5 for two indices that I have. But no change in memory usage.**

**How to make the search results faster ?**

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 5, 2019, 7:39am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/2 "2019-07-05T07:39:24Z")

</div>

Please don't post pictures of text, they are difficult to read, impossible to search and some people may not be even able to see them 🙂

> [@mrashid](#):
>
> Now, the problem is it takes long time to execute search in kibana

How long?

What is the output from `GET _cat/indices`?

---

<div class="post-metadata">

### Author: ![mrashid](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mrashid](https://discuss.elastic.co/u/mrashid)
#### Post date: [July 5, 2019, 7:48am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/3 "2019-07-05T07:48:25Z")

</div>

Thanks for responding back.  
It takes around 15-20 seconds .

I have got two indices . I just deleted the indices as I was playing with the shards and replicas. Anyways , here are the indices :

```
yellow open filebeat-failurelogs xctk6SElTImQipMLiTNDqw 15 5 6 0 78.4kb 78.4kb
green open .kibana aJQ8FbAbQU-S6EAfvN30Uw 1 0 2 0 15.8kb 15.8kb
yellow open filebeat-successlogs k3YK20_wRFO7MSd6WW-qQA 15 5 2615032 0 618.9mb 618.9mb

```

Both the indices collect no more than 2 Gb per day.

Should I update the Xms and Xmx in JVM options of ElasticSearch to around 70% of physical memory i.e around 40 GB ?

Do I need to do anything for Logstash and Kibana end ?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 5, 2019, 7:49am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/4 "2019-07-05T07:49:15Z")

</div>

You really only need 1 primary shard for 2GB of data, anymore is just wasteful and likely to cause slower response times, as the heap will be taken up managing mostly empty shards.

---

<div class="post-metadata">

### Author: ![mrashid](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mrashid](https://discuss.elastic.co/u/mrashid)
#### Post date: [July 5, 2019, 7:54am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/5 "2019-07-05T07:54:47Z")

</div>

Okay. Glad to know .

It will collect 2 GB per day , but will accumulate for 5 days to around 10-11 GB before the curator removes some of the data.

In that case is going with the default of 3 shards 2 replicas okay ?

---

<div class="post-metadata">

### Author: ![mrashid](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mrashid](https://discuss.elastic.co/u/mrashid)
#### Post date: [July 5, 2019, 7:57am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/6 "2019-07-05T07:57:52Z")

</div>

Let me go with 3 shards 2 replicas - the default setting and 40 GB max and min heap.  
i.e  
-Xms40g  
`-Xmx40g`

I will get back to inform the community of my findings.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [July 5, 2019, 8:13am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/7 "2019-07-05T08:13:01Z")

</div>

If you are only going to have that little data in the cluster, set the heap (Xms and Xmx) to 4GB. Then use a single primary shard per daily index. If you have more than 1 node, set the number of replica shards to 1 (or possibly 2 if you have 3 nodes in the cluster).

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 5, 2019, 8:18am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/8 "2019-07-05T08:18:06Z")

</div>

It might be easier to keep things to a single thread, as you also have this one going too - [Increase shards and replicas in new Index](https://discuss.elastic.co/t/increase-shards-and-replicas-in-new-index/188900/2)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 2, 2019, 8:18am UTC](https://discuss.elastic.co/t/terribly-slow-kibana-query/189010/9 "2019-08-02T08:18:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
