# Concurrent searches over same dataset degrades performance **URL:** \ **Category:** Elasticsearch\ **Created:** [April 17, 2015, 12:25pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293 "2015-04-17T12:25:36Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![vishal](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vishal](https://discuss.elastic.co/u/vishal)\ **Post date:** [April 17, 2015, 12:25pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/1 "2015-04-17T12:25:36Z")
Hi, I was trying Filtered query (default search type) to fetch first 8k out of approx 170k matched records. I noticed that on an average query took around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent searches over same dataset (in ideal scenario dataset will be different), I noticed that the performance degraded and each query took more than 1sec on average. Is this degradation in search response expected? I still haven't tried with scroll and scan as order is not important, but, curious to know if anything wrong with query or the way I executed. I am executing search on single index having around 850k docs (each doc contains around 5 fields), with 1g allocated to ES and using java node client to execute searches. Following is the query JSON: { "from" : 0, "size" : 8000, "query" : { "filtered" : { "query" : { "match\_all" : { } }, "filter" : { "and" : { "filters" : [ { "term" : { "details" : "Holiday" } }, { "range" : { "createdOn" : { "from" : 0, "to" : 1429260645351, "include\_lower" : true, "include\_upper" : true } } }, { "terms" : { "priority" : [1, 2] } } ] } } } }, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 17, 2015, 1:02pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/2 "2015-04-17T13:02:46Z")
What time do you measure? The ES query time, or the network latency? Jörg On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) wrote: > Hi, > > I was trying Filtered query (default search type) to fetch first 8k out of > approx 170k matched records. I noticed that on an average query took around > 500ms (response.getTookInMillis()). But, when I tried 4 concurrent searches > over same dataset (in ideal scenario dataset will be different), I noticed > that the performance degraded and each query took more than 1sec on > average. Is this degradation in search response expected? > I still haven't tried with scroll and scan as order is not important, but, > curious to know if anything wrong with query or the way I executed. > > I am executing search on single index having around 850k docs (each doc > contains around 5 fields), with 1g allocated to ES and using java node > client to execute searches. Following is the query JSON: > > { > "from" : 0, > "size" : 8000, > "query" : { > "filtered" : { > "query" : { > "match\_all" : { } > }, > "filter" : { > "and" : { > "filters" : [ { > "term" : { > "details" : "Holiday" > } > }, { > "range" : { > "createdOn" : { > "from" : 0, > "to" : 1429260645351, > "include\_lower" : true, > "include\_upper" : true > } > } > }, { > "terms" : { > "priority" : [1, 2] > } > } ] > } > } > } > }, > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEO8CNf1omLdiUSzrTNw4\_mXVrz\_wpEYsXBiPUiRH1U1w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEO8CNf1omLdiUSzrTNw4_mXVrz_wpEYsXBiPUiRH1U1w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 17, 2015, 1:04pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/3 "2015-04-17T13:04:17Z")
Sorry I overlooked it, you use getTookInMillis() Maybe the extra time is spent because you use a range filter which is not cached? Jörg On Fri, Apr 17, 2015 at 3:02 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > What time do you measure? The ES query time, or the network latency? > > Jörg > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) > wrote: > > > Hi, > > > > I was trying Filtered query (default search type) to fetch first 8k out > > of approx 170k matched records. I noticed that on an average query took > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > searches over same dataset (in ideal scenario dataset will be different), I > > noticed that the performance degraded and each query took more than 1sec on > > average. Is this degradation in search response expected? > > I still haven't tried with scroll and scan as order is not important, > > but, curious to know if anything wrong with query or the way I executed. > > > > I am executing search on single index having around 850k docs (each doc > > contains around 5 fields), with 1g allocated to ES and using java node > > client to execute searches. Following is the query JSON: > > > > { > > "from" : 0, > > "size" : 8000, > > "query" : { > > "filtered" : { > > "query" : { > > "match\_all" : { } > > }, > > "filter" : { > > "and" : { > > "filters" : [ { > > "term" : { > > "details" : "Holiday" > > } > > }, { > > "range" : { > > "createdOn" : { > > "from" : 0, > > "to" : 1429260645351, > > "include\_lower" : true, > > "include\_upper" : true > > } > > } > > }, { > > "terms" : { > > "priority" : [1, 2] > > } > > } ] > > } > > } > > } > > }, > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHcfFLER9\_zvxS7hC\_apQtJaageQHgU3CyCxqxuD\_asBQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHcfFLER9_zvxS7hC_apQtJaageQHgU3CyCxqxuD_asBQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vishal](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vishal](https://discuss.elastic.co/u/vishal)\ **Post date:** [April 17, 2015, 1:06pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/4 "2015-04-17T13:06:30Z")
ES query time as given by search response object (response.getTookInMillis()) Regards, Vishal On Friday, April 17, 2015 at 6:32:56 PM UTC+5:30, Jörg Prante wrote: > What time do you measure? The ES query time, or the network latency? > > Jörg > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan \<[vish...@gmail.com](mailto:vish...@gmail.com) > \\> wrote: > > > Hi, > > > > I was trying Filtered query (default search type) to fetch first 8k out > > of approx 170k matched records. I noticed that on an average query took > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > searches over same dataset (in ideal scenario dataset will be different), I > > noticed that the performance degraded and each query took more than 1sec on > > average. Is this degradation in search response expected? > > I still haven't tried with scroll and scan as order is not important, > > but, curious to know if anything wrong with query or the way I executed. > > > > I am executing search on single index having around 850k docs (each doc > > contains around 5 fields), with 1g allocated to ES and using java node > > client to execute searches. Following is the query JSON: > > > > { > > "from" : 0, > > "size" : 8000, > > "query" : { > > "filtered" : { > > "query" : { > > "match\_all" : { } > > }, > > "filter" : { > > "and" : { > > "filters" : [ { > > "term" : { > > "details" : "Holiday" > > } > > }, { > > "range" : { > > "createdOn" : { > > "from" : 0, > > "to" : 1429260645351, > > "include\_lower" : true, > > "include\_upper" : true > > } > > } > > }, { > > "terms" : { > > "priority" : [1, 2] > > } > > } ] > > } > > } > > } > > }, > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dd36ae7b-6c1a-404f-8b35-5359a302dddd%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd36ae7b-6c1a-404f-8b35-5359a302dddd%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vishal](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vishal](https://discuss.elastic.co/u/vishal)\ **Post date:** [April 17, 2015, 1:38pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/5 "2015-04-17T13:38:45Z")
I doubt that's the cause as it should also affect sequential searches. Regards, Vishal On Friday, April 17, 2015 at 6:34:24 PM UTC+5:30, Jörg Prante wrote: > Sorry I overlooked it, you use getTookInMillis() > > Maybe the extra time is spent because you use a range filter which is not > cached? > > Jörg > > On Fri, Apr 17, 2015 at 3:02 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) \ \< > [joerg...@gmail.com](mailto:joerg...@gmail.com) \\> wrote: > > > What time do you measure? The ES query time, or the network latency? > > > > Jörg > > > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan \<[vish...@gmail.com](mailto:vish...@gmail.com) > > \\> wrote: > > > > > Hi, > > > > > > I was trying Filtered query (default search type) to fetch first 8k out > > > of approx 170k matched records. I noticed that on an average query took > > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > > searches over same dataset (in ideal scenario dataset will be different), I > > > noticed that the performance degraded and each query took more than 1sec on > > > average. Is this degradation in search response expected? > > > I still haven't tried with scroll and scan as order is not important, > > > but, curious to know if anything wrong with query or the way I executed. > > > > > > I am executing search on single index having around 850k docs (each doc > > > contains around 5 fields), with 1g allocated to ES and using java node > > > client to execute searches. Following is the query JSON: > > > > > > { > > > "from" : 0, > > > "size" : 8000, > > > "query" : { > > > "filtered" : { > > > "query" : { > > > "match\_all" : { } > > > }, > > > "filter" : { > > > "and" : { > > > "filters" : [ { > > > "term" : { > > > "details" : "Holiday" > > > } > > > }, { > > > "range" : { > > > "createdOn" : { > > > "from" : 0, > > > "to" : 1429260645351, > > > "include\_lower" : true, > > > "include\_upper" : true > > > } > > > } > > > }, { > > > "terms" : { > > > "priority" : [1, 2] > > > } > > > } ] > > > } > > > } > > > } > > > }, > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 17, 2015, 1:55pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/6 "2015-04-17T13:55:59Z")
Do you round-robin the four concurrent searches over the cluster nodes? Jörg On Fri, Apr 17, 2015 at 3:38 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) wrote: > I doubt that's the cause as it should also affect sequential searches. > > Regards, > Vishal > > On Friday, April 17, 2015 at 6:34:24 PM UTC+5:30, Jörg Prante wrote: > > > Sorry I overlooked it, you use getTookInMillis() > > > > Maybe the extra time is spent because you use a range filter which is not > > cached? > > > > Jörg > > > > On Fri, Apr 17, 2015 at 3:02 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) [joerg...@gmail.com](mailto:joerg...@gmail.com) > > wrote: > > > > > What time do you measure? The ES query time, or the network latency? > > > > > > Jörg > > > > > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan [vish...@gmail.com](mailto:vish...@gmail.com) > > > wrote: > > > > > > > Hi, > > > > > > > > I was trying Filtered query (default search type) to fetch first 8k out > > > > of approx 170k matched records. I noticed that on an average query took > > > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > > > searches over same dataset (in ideal scenario dataset will be different), I > > > > noticed that the performance degraded and each query took more than 1sec on > > > > average. Is this degradation in search response expected? > > > > I still haven't tried with scroll and scan as order is not important, > > > > but, curious to know if anything wrong with query or the way I executed. > > > > > > > > I am executing search on single index having around 850k docs (each doc > > > > contains around 5 fields), with 1g allocated to ES and using java node > > > > client to execute searches. Following is the query JSON: > > > > > > > > { > > > > "from" : 0, > > > > "size" : 8000, > > > > "query" : { > > > > "filtered" : { > > > > "query" : { > > > > "match\_all" : { } > > > > }, > > > > "filter" : { > > > > "and" : { > > > > "filters" : [ { > > > > "term" : { > > > > "details" : "Holiday" > > > > } > > > > }, { > > > > "range" : { > > > > "createdOn" : { > > > > "from" : 0, > > > > "to" : 1429260645351, > > > > "include\_lower" : true, > > > > "include\_upper" : true > > > > } > > > > } > > > > }, { > > > > "terms" : { > > > > "priority" : [1, 2] > > > > } > > > > } ] > > > > } > > > > } > > > > } > > > > }, > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vishal](https://avatars.discourse-cdn.com/v4/letter/v/49beb7/32.png) [@vishal](https://discuss.elastic.co/u/vishal)\ **Post date:** [April 17, 2015, 2:21pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/7 "2015-04-17T14:21:48Z")
I have single node cluster. Not sure what you mean by round robin in concurrent searches. Regards, Vishal On Apr 17, 2015 7:26 PM, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Do you round-robin the four concurrent searches over the cluster nodes? > > Jörg > > On Fri, Apr 17, 2015 at 3:38 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) > wrote: > > > I doubt that's the cause as it should also affect sequential searches. > > > > Regards, > > Vishal > > > > On Friday, April 17, 2015 at 6:34:24 PM UTC+5:30, Jörg Prante wrote: > > > > > Sorry I overlooked it, you use getTookInMillis() > > > > > > Maybe the extra time is spent because you use a range filter which is > > > not cached? > > > > > > Jörg > > > > > > On Fri, Apr 17, 2015 at 3:02 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) [joerg...@gmail.com](mailto:joerg...@gmail.com) > > > wrote: > > > > > > > What time do you measure? The ES query time, or the network latency? > > > > > > > > Jörg > > > > > > > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan [vish...@gmail.com](mailto:vish...@gmail.com) > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I was trying Filtered query (default search type) to fetch first 8k > > > > > out of approx 170k matched records. I noticed that on an average query took > > > > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > > > > searches over same dataset (in ideal scenario dataset will be different), I > > > > > noticed that the performance degraded and each query took more than 1sec on > > > > > average. Is this degradation in search response expected? > > > > > I still haven't tried with scroll and scan as order is not important, > > > > > but, curious to know if anything wrong with query or the way I executed. > > > > > > > > > > I am executing search on single index having around 850k docs (each > > > > > doc contains around 5 fields), with 1g allocated to ES and using java node > > > > > client to execute searches. Following is the query JSON: > > > > > > > > > > { > > > > > "from" : 0, > > > > > "size" : 8000, > > > > > "query" : { > > > > > "filtered" : { > > > > > "query" : { > > > > > "match\_all" : { } > > > > > }, > > > > > "filter" : { > > > > > "and" : { > > > > > "filters" : [ { > > > > > "term" : { > > > > > "details" : "Holiday" > > > > > } > > > > > }, { > > > > > "range" : { > > > > > "createdOn" : { > > > > > "from" : 0, > > > > > "to" : 1429260645351, > > > > > "include\_lower" : true, > > > > > "include\_upper" : true > > > > > } > > > > > } > > > > > }, { > > > > > "terms" : { > > > > > "priority" : [1, 2] > > > > > } > > > > > } ] > > > > > } > > > > > } > > > > > } > > > > > }, > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/jzK56eE8Pd0/unsubscribe](https://groups.google.com/d/topic/elasticsearch/jzK56eE8Pd0/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN\_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 17, 2015, 2:49pm UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/8 "2015-04-17T14:49:11Z")
You can not expect a single node cluster can work faster when being searched concurrently. Four concurrent searches require four times the resources, such as CPU and memory. Jörg On Fri, Apr 17, 2015 at 4:21 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) wrote: > I have single node cluster. Not sure what you mean by round robin in > concurrent searches. > > Regards, > Vishal > On Apr 17, 2015 7:26 PM, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > > > Do you round-robin the four concurrent searches over the cluster nodes? > > > > Jörg > > > > On Fri, Apr 17, 2015 at 3:38 PM, Vishal Mahajan [vishal423@gmail.com](mailto:vishal423@gmail.com) > > wrote: > > > > > I doubt that's the cause as it should also affect sequential searches. > > > > > > Regards, > > > Vishal > > > > > > On Friday, April 17, 2015 at 6:34:24 PM UTC+5:30, Jörg Prante wrote: > > > > > > > Sorry I overlooked it, you use getTookInMillis() > > > > > > > > Maybe the extra time is spent because you use a range filter which is > > > > not cached? > > > > > > > > Jörg > > > > > > > > On Fri, Apr 17, 2015 at 3:02 PM, [joerg...@gmail.com](mailto:joerg...@gmail.com) \<[joerg...@gmail.com](mailto:joerg...@gmail.com) > > > > > > > > > wrote: > > > > > > > > > What time do you measure? The ES query time, or the network latency? > > > > > > > > > > Jörg > > > > > > > > > > On Fri, Apr 17, 2015 at 2:25 PM, Vishal Mahajan [vish...@gmail.com](mailto:vish...@gmail.com) > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I was trying Filtered query (default search type) to fetch first 8k > > > > > > out of approx 170k matched records. I noticed that on an average query took > > > > > > around 500ms (response.getTookInMillis()). But, when I tried 4 concurrent > > > > > > searches over same dataset (in ideal scenario dataset will be different), I > > > > > > noticed that the performance degraded and each query took more than 1sec on > > > > > > average. Is this degradation in search response expected? > > > > > > I still haven't tried with scroll and scan as order is not important, > > > > > > but, curious to know if anything wrong with query or the way I executed. > > > > > > > > > > > > I am executing search on single index having around 850k docs (each > > > > > > doc contains around 5 fields), with 1g allocated to ES and using java node > > > > > > client to execute searches. Following is the query JSON: > > > > > > > > > > > > { > > > > > > "from" : 0, > > > > > > "size" : 8000, > > > > > > "query" : { > > > > > > "filtered" : { > > > > > > "query" : { > > > > > > "match\_all" : { } > > > > > > }, > > > > > > "filter" : { > > > > > > "and" : { > > > > > > "filters" : [ { > > > > > > "term" : { > > > > > > "details" : "Holiday" > > > > > > } > > > > > > }, { > > > > > > "range" : { > > > > > > "createdOn" : { > > > > > > "from" : 0, > > > > > > "to" : 1429260645351, > > > > > > "include\_lower" : true, > > > > > > "include\_upper" : true > > > > > > } > > > > > > } > > > > > > }, { > > > > > > "terms" : { > > > > > > "priority" : [1, 2] > > > > > > } > > > > > > } ] > > > > > > } > > > > > > } > > > > > > } > > > > > > }, > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d4075dd3-f711-458e-a89c-522f32fa3e18%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/32c073c6-bef8-465c-8e70-de55a1e195c8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/jzK56eE8Pd0/unsubscribe](https://groups.google.com/d/topic/elasticsearch/jzK56eE8Pd0/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE2-7kSh7t892QXEVYhqU5g6TnTVAZR-hb5%3Ds07xajMdw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN\_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN\_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAG2s5vN_5GGXexCCh4sBXKeoPDcdogPh18fCQwbLrrR2hvxXMg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEHdCfk2NwSHjWSGDS31LFW2zAVNoe2Hz4aBLEHz%2BDRZw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEHdCfk2NwSHjWSGDS31LFW2zAVNoe2Hz4aBLEHz%2BDRZw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:19am UTC](https://discuss.elastic.co/t/concurrent-searches-over-same-dataset-degrades-performance/23293/9 "2017-07-06T00:19:01Z")