# ES slow query problem **URL:** \ **Category:** Elasticsearch\ **Created:** [October 19, 2012, 7:43pm UTC](https://discuss.elastic.co/t/es-slow-query-problem/9424 "2012-10-19T19:43:42Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [October 19, 2012, 7:43pm UTC](https://discuss.elastic.co/t/es-slow-query-problem/9424/1 "2012-10-19T19:43:42Z")
hi, im trying to figure out how to speed up my queries.. nothing is helping.. i have a cluster of 3 nodes with 1 replica (2 data nodes and 1 non-data LB node). each of the 2 data nodes are m1.xlarge on ec2 with 8GB of ram mlocked on each of them. i am logging slow queries.. there are mainly 2 indices with 5 shards each. what are my options to resolve this? here is an example of slow query log - [2012-10-19 19:30:30,570][WARN][index.search.slowlog.fetch] [Phimster, Ellie] [ twitter][2] took[3s], took\_millis[3010], search\_type[QUERY\_THEN\_FETCH], total\_sh ards[5], source[{"size":100,"sort":{"score":{"order":"desc"},"\_score":{}},"query ":{"query\_string":{"fields":["text","product\_categories"],"query":"SWAROVSKI ELE MENTS OR Swarovski Rhinestones OR Rhinestone Shapes OR 2300 Teardrop"}},"filter ":{"terms":{"brand\_id":["14956545"],"minimum\_match":1}}}extra\_source[], most of our queries are filtered on a field in the document. but we are not using routing feature yet. the problem with using routing feature now is what happens to data that is already indexed on other shards? migrating/moving will be a huge deal i think. any advice would be helpful. thanks -- ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [October 20, 2012, 3:01am UTC](https://discuss.elastic.co/t/es-slow-query-problem/9424/2 "2012-10-20T03:01:51Z")
Hi, Questions: - how big are your indices? - what Xmx are you using? - is there disk IO? - is this right after start or after caches have been warmed? - if the disk is constantly being updated, try increasing refresh interval - what does your system/ES monitoring tool show you? Do you really want .... OR Swarovski Rhinestones OR Rhinestone Shapes ... or do you actually want ... OR "Swarovski Rhinestones" OR "Rhinestone Shapes"... ? ## Otis Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) Performance Monitoring - [Sematext Monitoring | Infrastructure Monitoring Service](http://sematext.com/spm/index.html) On Friday, October 19, 2012 3:43:51 PM UTC-4, T Vinod Gupta wrote: > hi, > im trying to figure out how to speed up my queries.. nothing is helping.. > i have a cluster of 3 nodes with 1 replica (2 data nodes and 1 non-data LB > node). each of the 2 data nodes are m1.xlarge on ec2 with 8GB of ram > mlocked on each of them. i am logging slow queries.. there are mainly 2 > indices with 5 shards each. what are my options to resolve this? > > here is an example of slow query log - > [2012-10-19 19:30:30,570][WARN][index.search.slowlog.fetch] [Phimster, > Ellie] [ > twitter][2] took[3s], took\_millis[3010], search\_type[QUERY\_THEN\_FETCH], > total\_sh > ards[5], > source[{"size":100,"sort":{"score":{"order":"desc"},"\_score":{}},"query > ":{"query\_string":{"fields":["text","product\_categories"],"query":"SWAROVSKI > ELE > MENTS OR Swarovski Rhinestones OR Rhinestone Shapes OR 2300 > Teardrop"}},"filter > ":{"terms":{"brand\_id":["14956545"],"minimum\_match":1}}}extra\_source, > > most of our queries are filtered on a field in the document. but we are > not using routing feature yet. the problem with using routing feature now > is what happens to data that is already indexed on other shards? > migrating/moving will be a huge deal i think. > > any advice would be helpful. > > thanks -- ---
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [October 21, 2012, 4:36pm UTC](https://discuss.elastic.co/t/es-slow-query-problem/9424/3 "2012-10-21T16:36:07Z")
i was able to make some headway by using filtered queries (instead of query with a filter). there is a subtle difference between the two (we dont use facets) but i believe big difference in perf. to answer your questions - indices are about 50GB total, Xmx/Xms is 8GB. this is in the stable state. refresh interval is set to 60 sec. bigdesk says about 50% of allocated heap is used, rest is free. threads are around 125 with peak at 160. regarding the text query, we really want best phrase match.. but if thats not possible, match on the words inside the phrases. that part is probably not fully correct. thanks On Fri, Oct 19, 2012 at 8:01 PM, Otis Gospodnetic \< [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote: > Hi, > > Questions: > > - how big are your indices? > - what Xmx are you using? > - is there disk IO? > - is this right after start or after caches have been warmed? > - if the disk is constantly being updated, try increasing refresh interval > - what does your system/ES monitoring tool show you? > > Do you really want .... OR Swarovski Rhinestones OR Rhinestone Shapes ... > > or do you actually want ... OR "Swarovski Rhinestones" OR "Rhinestone > Shapes"... > > ? > > ## Otis > > Search Analytics - [http://sematext.com/search-\*\*analytics/index.html](http://sematext.com/search-**analytics/index.html)[http://sematext.com/search-analytics/index.html](http://sematext.com/search-analytics/index.html) > Performance Monitoring - [http://sematext.com/spm/index.\*\*html](http://sematext.com/spm/index.**html)[http://sematext.com/spm/index.html](http://sematext.com/spm/index.html) > > On Friday, October 19, 2012 3:43:51 PM UTC-4, T Vinod Gupta wrote: > > > hi, > > im trying to figure out how to speed up my queries.. nothing is helping.. > > i have a cluster of 3 nodes with 1 replica (2 data nodes and 1 non-data LB > > node). each of the 2 data nodes are m1.xlarge on ec2 with 8GB of ram > > mlocked on each of them. i am logging slow queries.. there are mainly 2 > > indices with 5 shards each. what are my options to resolve this? > > > > here is an example of slow query log - > > [2012-10-19 19:30:30,570][WARN][index.search.slowlog.fetch] [Phimster, > > Ellie] [ > > twitter][2] took[3s], took\_millis[3010], search\_type[QUERY\_THEN\_FETCH], > > total\_sh > > ards[5], source[{"size":100,"sort":{" **score":{"order":"desc"},"\_** > > score":{}},"query > > ":{"query\_string":{"fields":["\*\*text","product\_categories"],"\*\*query":"SWAROVSKI > > ELE > > MENTS OR Swarovski Rhinestones OR Rhinestone Shapes OR 2300 > > Teardrop"}},"filter > > ":{"terms":{"brand\_id":["**14956545"],"minimum\_match":1}}** > > }extra\_source, > > > > most of our queries are filtered on a field in the document. but we are > > not using routing feature yet. the problem with using routing feature now > > is what happens to data that is already indexed on other shards? > > migrating/moving will be a huge deal i think. > > > > any advice would be helpful. > > > > thanks > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:07am UTC](https://discuss.elastic.co/t/es-slow-query-problem/9424/4 "2017-07-06T03:07:43Z")