# Facing issue with occasional slow queries on 5 node ES cluster
**URL:** https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228
**Category:** Elasticsearch
**Created:** [April 5, 2021, 11:50am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228 "2021-04-05T11:50:46Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 5, 2021, 11:50am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/1 "2021-04-05T11:50:46Z")
I have a 5 node ES cluster and it's performing well on an average but sometime 0.5% of the queries are taking 1 sec to respond. I have tried all the possible solutions/suggestions, not sure what is causing issue.
Overview of cluster:
3 Master with 4gb allocated heap out of 8 gb RAM
2 **data node with 4 gb allocated heap out of 64 gb RAM and 70 % of heap is utilized, should i consider increasing the heap size ( Average throughput is 15000 tps).**
On all the master OS mem is 94 % used.
Please suggest what should i do to handle 15000 tps with no slow queries.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 6, 2021, 1:11am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/2 "2021-04-06T01:11:37Z")
Welcome to our community! 😃
> [@Vikash](#):
>
> 2 **data node with 4 gb allocated heap out of 64 gb RAM and 70 % of heap is utilized, should i consider increasing the heap size ( Average throughput is 15000 tps).**
That'd be a start.
Otherwise what does hot threads show at that time? Or slow log? Do you have Monitoring enabled? What version are you on?
---
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 6, 2021, 5:15am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/3 "2021-04-06T05:15:27Z")
Thanks mark for quick response.
Actually we have set threshold for slowlog as 5 ms as we want max response time to be 5 ms for each queries. But 90% of the queries are below threshold and few queries have been logged in slow log. Here some 0.5% of the queries are taking longer time (1 sec). we are using 7.10 version.
One more observation i have is on each master node os memory is 96% used so, it it fine or we need to allocate more ?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 6, 2021, 5:18am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/4 "2021-04-06T05:18:18Z")
> [@Vikash](#):
>
> One more observation i have is on each master node os memory is 96% used so, it it fine or we need to allocate more ?
Is that system memory or heap?
---
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 6, 2021, 5:18am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/5 "2021-04-06T05:18:54Z")
it's RAM
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 6, 2021, 5:21am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/6 "2021-04-06T05:21:10Z")
So system, in that case it's not a major concern.
If your heap on the data nodes is 4 gig and you have 64 gig, then increasing heap would be worth doing.
---
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 6, 2021, 5:22am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/7 "2021-04-06T05:22:50Z")
ok.. I will try that ..Thanks
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 6, 2021, 5:42am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/8 "2021-04-06T05:42:11Z")
What is the full output of the [cat nodes API](https://www.elastic.co/guide/en/elasticsearch/reference/7.12/cat-nodes.html) and the [cluster stats API](https://www.elastic.co/guide/en/elasticsearch/reference/7.12/cluster-stats.html)?
---
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 7, 2021, 6:20am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/9 "2021-04-07T06:20:07Z")
Christian and Mark, should i consider putting more heap for master as well , i have kept it 4 gig thinking master does n't do much compute operations.
What kind of machine instance we should be using for master and data nodes for ex. General purpose machine, m/m optimized . CPU optimized .. Please suggest
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 7, 2021, 8:14am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/10 "2021-04-07T08:14:15Z")
Are they master only nodes?
---
### Author:  [@Vikash](https://discuss.elastic.co/u/Vikash)
#### Post date: [April 7, 2021, 8:47am UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/11 "2021-04-07T08:47:12Z")
Yes..they are dedicated master
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 7, 2021, 9:04pm UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/12 "2021-04-07T21:04:57Z")
Then that should be fine, master nodes don't need a lot.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 5, 2021, 9:05pm UTC](https://discuss.elastic.co/t/facing-issue-with-occasional-slow-queries-on-5-node-es-cluster/269228/13 "2021-05-05T21:05:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.