# How does non-data node choose a node for query/fetch? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 29, 2012, 10:48pm UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497 "2012-10-29T22:48:48Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [October 29, 2012, 10:48pm UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/1 "2012-10-29T22:48:48Z")
when a non-data node receives a search request, how does it go about choosing the nodes that have replicas of a shard? e.g if number of replicas is 1 and there are 2 data nodes then both nodes have complete index data and shards. if there is a non-data node fronting these 2 nodes and it gets a search request, how does it distribute the work? is there a setting to make it fire parallel redundant requests to both nodes and return the results as soon as they are ready (without waiting for both nodes to respond)? i guess the situation im trying to address is a case where a slow node in a cluster bottlenecks the cluster as a whole. thanks -- ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [October 30, 2012, 1:27pm UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/2 "2012-10-30T13:27:18Z")
Hello, As far as I know, a round robin is used to distribute searches between replicas. So if a node is slower, queries hitting shards on that node would normally be slower. My understanding is that adding replicas help with concurrent queries. If you run one query at a time, your searches should be just as fast. Regarding the situation your trying to address, I think a solution might be to divide your data into multiple indices - based on how "hot" that data is. So I'd put data that is queried more frequently on the faster nodes, using shard allocation filtering: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... And you have to make sure that, within your application, you query only the needed indices, not all of them. ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene On Tue, Oct 30, 2012 at 12:48 AM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com) wrote: > when a non-data node receives a search request, how does it go about > choosing the nodes that have replicas of a shard? e.g if number of replicas > is 1 and there are 2 data nodes then both nodes have complete index data and > shards. if there is a non-data node fronting these 2 nodes and it gets a > search request, how does it distribute the work? is there a setting to make > it fire parallel redundant requests to both nodes and return the results as > soon as they are ready (without waiting for both nodes to respond)? > > i guess the situation im trying to address is a case where a slow node in a > cluster bottlenecks the cluster as a whole. > > thanks -- ---
**Author:** ![T\_Vinod\_Gupta](https://avatars.discourse-cdn.com/v4/letter/t/fbc32d/32.png) [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)\ **Post date:** [November 1, 2012, 2:03am UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/3 "2012-11-01T02:03:34Z")
thanks for the info.. now isnt that inefficient as it doesnt take advantage of caching.. lets say 1st search goes to node1 and it caches the filter results. same search is fired again.. if it goes to node2, then you have lost cache advantage. am i missing something? thanks On Tue, Oct 30, 2012 at 6:27 AM, Radu Gheorghe [radu.gheorghe@sematext.com](mailto:radu.gheorghe@sematext.com)wrote: > Hello, > > As far as I know, a round robin is used to distribute searches between > replicas. So if a node is slower, queries hitting shards on that node > would normally be slower. > > My understanding is that adding replicas help with concurrent queries. > If you run one query at a time, your searches should be just as fast. > > Regarding the situation your trying to address, I think a solution > might be to divide your data into multiple indices - based on how > "hot" that data is. So I'd put data that is queried more frequently on > the faster nodes, using shard allocation filtering: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > And you have to make sure that, within your application, you query > only the needed indices, not all of them. > > ## Best regards, Radu > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > On Tue, Oct 30, 2012 at 12:48 AM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com) > wrote: > > > when a non-data node receives a search request, how does it go about > > choosing the nodes that have replicas of a shard? e.g if number of > > replicas > > is 1 and there are 2 data nodes then both nodes have complete index data > > and > > shards. if there is a non-data node fronting these 2 nodes and it gets a > > search request, how does it distribute the work? is there a setting to > > make > > it fire parallel redundant requests to both nodes and return the results > > as > > soon as they are ready (without waiting for both nodes to respond)? > > > > i guess the situation im trying to address is a case where a slow node > > in a > > cluster bottlenecks the cluster as a whole. > > > > thanks > > -- -- ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [November 1, 2012, 7:52am UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/4 "2012-11-01T07:52:11Z")
Hello, On Thu, Nov 1, 2012 at 4:03 AM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com) wrote: > thanks for the info.. > now isnt that inefficient as it doesnt take advantage of caching.. lets say > 1st search goes to node1 and it caches the filter results. same search is > fired again.. if it goes to node2, then you have lost cache advantage. > am i missing something? Well, if you run a lot of queries, all nodes should warm up and you shouldn't notice a difference. If it's not the case, you can use Search Preferences to prefer running on the primary shards: > **[Elastic — The Search AI Company](https://www.elastic.co)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 1, 2012, 10:40am UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/5 "2012-11-01T10:40:22Z")
You can also use "custom value" in search preference to make sure that all requests that contain the same search stick to the same shard (not necessary primary). However, in this case you will have to balance the load between shards yourself. On Thursday, November 1, 2012 3:52:14 AM UTC-4, Radu Gheorghe wrote: > Hello, > > On Thu, Nov 1, 2012 at 4:03 AM, T Vinod Gupta \<[tvi...@readypulse.com](mailto:tvi...@readypulse.com)\\> > wrote: > > > thanks for the info.. > > now isnt that inefficient as it doesnt take advantage of caching.. lets > > say > > 1st search goes to node1 and it caches the filter results. same search > > is > > fired again.. if it goes to node2, then you have lost cache advantage. > > am i missing something? > > Well, if you run a lot of queries, all nodes should warm up and you > shouldn't notice a difference. If it's not the case, you can use > Search Preferences to prefer running on the primary shards: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html) > > ## Best regards, Radu > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene -- ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [November 1, 2012, 7:07pm UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/6 "2012-11-01T19:07:06Z")
Hi Vinod, Indeed, it is ineffcient and Radu and Igor already provided solutions. You can see the inefficiency in a multi-node cluster (heh, a bit of word redundancy never hurt anyone, I suppose) where you run the same query a couple of times. You will see higher latency initially, but after a few requests the latency will drop due to caches. Note that it's not only about filter caches or any other ES-level caching, but also OS caches. ## Otis Search Analytics - [Cloud Monitoring Tools & Services | Sematext](http://sematext.com/search-analytics/index.html) Performance Monitoring - [http://sematext.com/spm/index](http://sematext.com/spm/index)[http://sematext.com/spm/index.html](http://sematext.com/spm/index.html) On Wednesday, October 31, 2012 10:03:44 PM UTC-4, T Vinod Gupta wrote: > thanks for the info.. > now isnt that inefficient as it doesnt take advantage of caching.. lets > say 1st search goes to node1 and it caches the filter results. same search > is fired again.. if it goes to node2, then you have lost cache advantage. > am i missing something? > > thanks > > On Tue, Oct 30, 2012 at 6:27 AM, Radu Gheorghe \<[radu.g...@sematext.com](mailto:radu.g...@sematext.com)\ > > > wrote: > > > Hello, > > > > As far as I know, a round robin is used to distribute searches between > > replicas. So if a node is slower, queries hitting shards on that node > > would normally be slower. > > > > My understanding is that adding replicas help with concurrent queries. > > If you run one query at a time, your searches should be just as fast. > > > > Regarding the situation your trying to address, I think a solution > > might be to divide your data into multiple indices - based on how > > "hot" that data is. So I'd put data that is queried more frequently on > > the faster nodes, using shard allocation filtering: > > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > > > And you have to make sure that, within your application, you query > > only the needed indices, not all of them. > > > > ## Best regards, Radu > > > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > > > On Tue, Oct 30, 2012 at 12:48 AM, T Vinod Gupta \<[tvi...@readypulse.com](mailto:tvi...@readypulse.com)\\> > > wrote: > > > > > when a non-data node receives a search request, how does it go about > > > choosing the nodes that have replicas of a shard? e.g if number of > > > replicas > > > is 1 and there are 2 data nodes then both nodes have complete index > > > data and > > > shards. if there is a non-data node fronting these 2 nodes and it gets a > > > search request, how does it distribute the work? is there a setting to > > > make > > > it fire parallel redundant requests to both nodes and return the > > > results as > > > soon as they are ready (without waiting for both nodes to respond)? > > > > > > i guess the situation im trying to address is a case where a slow node > > > in a > > > cluster bottlenecks the cluster as a whole. > > > > > > thanks > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:06am UTC](https://discuss.elastic.co/t/how-does-non-data-node-choose-a-node-for-query-fetch/9497/7 "2017-07-06T03:06:31Z")