# Improving a slow running Match\_All Query **URL:** \ **Category:** Elasticsearch\ **Created:** [May 28, 2014, 11:10pm UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792 "2014-05-28T23:10:26Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![sairam\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sairam_2/32/1475_2.png) [@sairam\_2](https://discuss.elastic.co/u/sairam_2)\ **Post date:** [May 28, 2014, 11:10pm UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792/1 "2014-05-28T23:10:26Z")
Hello, The queries that we run seem to be very CPU Intensive and cause the Servers to max out within a short amount of time. On debugging, it looks like standard queries take too long to respond too. We are currently running version 1.0.2 of Elasticsearch and have about 67.3G of data on Production. There are currently 5 Shards running on 2 Nodes (1 Replica). There is a total of 252gb RAM with Heap Size set to 109.9gb. [https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX\_\_I/AAAAAAAAAAM/XA8B0kff\_u8/s1600/Elasticsearch+Setup.png](https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX__I/AAAAAAAAAAM/XA8B0kff_u8/s1600/Elasticsearch+Setup.png) The Indexing rate is high since we are migrating data: From Bigdesk: [https://lh4.googleusercontent.com/-gwJ\_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png](https://lh4.googleusercontent.com/-gwJ_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png) The Refresh Activity report from ElasticHQ: [https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png](https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png) The Match All Query takes a whopping 520 - 680ms to run. { "query": { "match\_all": {} } } However, on a similar Test Environment Setup (with 8G of data), the same query takes about 80-120ms to execute. Which feels more like the average. [https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt\_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png](https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png) What are some of the recommendations that can improve this bottleneck? Will adding more Nodes help alleviate this issue or will it worsen it. Thanks, Sairam -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/66c26a89-b3f8-4a8c-96dc-45babc1e012d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66c26a89-b3f8-4a8c-96dc-45babc1e012d%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![sairam\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sairam_2/32/1475_2.png) [@sairam\_2](https://discuss.elastic.co/u/sairam_2)\ **Post date:** [May 30, 2014, 7:20pm UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792/2 "2014-05-30T19:20:26Z")
_Bump_ On Wednesday, May 28, 2014 4:10:26 PM UTC-7, [sai...@roblox.com](mailto:sai...@roblox.com) wrote: > Hello, > > The queries that we run seem to be very CPU Intensive and cause the > Servers to max out within a short amount of time. On debugging, it looks > like standard queries take too long to respond too. > > We are currently running version 1.0.2 of Elasticsearch and have about > 67.3G of data on Production. There are currently 5 Shards running on 2 > Nodes (1 Replica). There is a total of 252gb RAM with Heap Size set to > 109.9gb. > > [https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX\_\_I/AAAAAAAAAAM/XA8B0kff\_u8/s1600/Elasticsearch+Setup.png](https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX__I/AAAAAAAAAAM/XA8B0kff_u8/s1600/Elasticsearch+Setup.png) > > The Indexing rate is high since we are migrating data: > > From Bigdesk: > > [https://lh4.googleusercontent.com/-gwJ\_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png](https://lh4.googleusercontent.com/-gwJ_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png) > > The Refresh Activity report from ElasticHQ: > > [https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png](https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png) > > The Match All Query takes a whopping 520 - 680ms to run. > { > "query": { > "match\_all": {} > } > } > > However, on a similar Test Environment Setup (with 8G of data), the same > query takes about 80-120ms to execute. Which feels more like the average. > > [https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt\_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png](https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png) > What are some of the recommendations that can improve this bottleneck? Will > adding more Nodes help alleviate this issue or will it worsen it. > > Thanks, > Sairam -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [May 30, 2014, 8:52pm UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792/3 "2014-05-30T20:52:39Z")
Is "match\_all" always running at that time or is it getting faster after a first run? Did you run an optimize with maximum number of segments? What is your segment count? Jörg On Fri, May 30, 2014 at 9:20 PM, [sairam@roblox.com](mailto:sairam@roblox.com) wrote: > _Bump_ > > On Wednesday, May 28, 2014 4:10:26 PM UTC-7, [sai...@roblox.com](mailto:sai...@roblox.com) wrote: > > > Hello, > > > > The queries that we run seem to be very CPU Intensive and cause the > > Servers to max out within a short amount of time. On debugging, it looks > > like standard queries take too long to respond too. > > > > We are currently running version 1.0.2 of Elasticsearch and have about > > 67.3G of data on Production. There are currently 5 Shards running on 2 > > Nodes (1 Replica). There is a total of 252gb RAM with Heap Size set to > > 109.9gb. > > > > [https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX\_\_I/AAAAAAAAAAM/XA8B0kff\_u8/s1600/Elasticsearch+Setup.png](https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX__I/AAAAAAAAAAM/XA8B0kff_u8/s1600/Elasticsearch+Setup.png) > > > > The Indexing rate is high since we are migrating data: > > > > From Bigdesk: > > > > [https://lh4.googleusercontent.com/-gwJ\_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png](https://lh4.googleusercontent.com/-gwJ_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png) > > > > The Refresh Activity report from ElasticHQ: > > > > [https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png](https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png) > > > > The Match All Query takes a whopping 520 - 680ms to run. > > { > > "query": { > > "match\_all": {} > > } > > } > > > > However, on a similar Test Environment Setup (with 8G of data), the same > > query takes about 80-120ms to execute. Which feels more like the average. > > > > [https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt\_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png](https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png) > > What are some of the recommendations that can improve this bottleneck? Will > > adding more Nodes help alleviate this issue or will it worsen it. > > > > Thanks, > > Sairam > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE57R%2BUnx-xUf9%3D0bb76e\_JBzsycrsiFDiC%2BDujAus-zw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE57R%2BUnx-xUf9%3D0bb76e_JBzsycrsiFDiC%2BDujAus-zw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![sairam\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sairam_2/32/1475_2.png) [@sairam\_2](https://discuss.elastic.co/u/sairam_2)\ **Post date:** [May 30, 2014, 9:03pm UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792/4 "2014-05-30T21:03:05Z")
Yes, the match\_all keeps taking that time. It hasn't improved after the first few queries. I did not run the Optimize command since we were in the middle of Indexing. I can run it now by setting the max\_num\_segments to 1. On Friday, May 30, 2014 1:52:55 PM UTC-7, Jörg Prante wrote: > Is "match\_all" always running at that time or is it getting faster after a > first run? > > Did you run an optimize with maximum number of segments? What is your > segment count? > > Jörg > > On Fri, May 30, 2014 at 9:20 PM, \<[sai...@roblox.com](mailto:sai...@roblox.com) \\> wrote: > > > _Bump_ > > > > On Wednesday, May 28, 2014 4:10:26 PM UTC-7, [sai...@roblox.com](mailto:sai...@roblox.com) wrote: > > > > > Hello, > > > > > > The queries that we run seem to be very CPU Intensive and cause the > > > Servers to max out within a short amount of time. On debugging, it looks > > > like standard queries take too long to respond too. > > > > > > We are currently running version 1.0.2 of Elasticsearch and have about > > > 67.3G of data on Production. There are currently 5 Shards running on 2 > > > Nodes (1 Replica). There is a total of 252gb RAM with Heap Size set to > > > 109.9gb. > > > > > > [https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX\_\_I/AAAAAAAAAAM/XA8B0kff\_u8/s1600/Elasticsearch+Setup.png](https://lh5.googleusercontent.com/-4bLBVeQuIHY/U4YnR7KX__I/AAAAAAAAAAM/XA8B0kff_u8/s1600/Elasticsearch+Setup.png) > > > > > > The Indexing rate is high since we are migrating data: > > > > > > From Bigdesk: > > > > > > [https://lh4.googleusercontent.com/-gwJ\_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png](https://lh4.googleusercontent.com/-gwJ_8GUT5nc/U4ZplswvYYI/AAAAAAAAAA0/6UhKvCpCF6c/s1600/Indexing+Rate.png) > > > > > > The Refresh Activity report from ElasticHQ: > > > > > > [https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png](https://lh6.googleusercontent.com/-DHO2pXXflME/U4ZqLxawJKI/AAAAAAAAABA/7pqxXnKbLn0/s1600/ElasticHQ+-+Index+Activity.png) > > > > > > The Match All Query takes a whopping 520 - 680ms to run. > > > { > > > "query": { > > > "match\_all": {} > > > } > > > } > > > > > > However, on a similar Test Environment Setup (with 8G of data), the same > > > query takes about 80-120ms to execute. Which feels more like the average. > > > > > > [https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt\_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png](https://lh5.googleusercontent.com/-LTtjBlbcHR0/U4YqUTlNt_I/AAAAAAAAAAc/P1h6okpdm4w/s1600/Elasticsearch+Test+Setup.png) > > > What are some of the recommendations that can improve this bottleneck? Will > > > adding more Nodes help alleviate this issue or will it worsen it. > > > > > > Thanks, > > > Sairam > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/30267e6d-317b-4bb2-aa8a-66f05cfdf49f%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f5f590d4-649d-4147-a059-270e1cf2320f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f5f590d4-649d-4147-a059-270e1cf2320f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:25am UTC](https://discuss.elastic.co/t/improving-a-slow-running-match-all-query/17792/5 "2017-07-06T01:25:37Z")