# ElasticSearch number of shards queried **URL:** \ **Category:** Elasticsearch\ **Created:** [April 16, 2019, 7:18am UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011 "2019-04-16T07:18:52Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 7:18am UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/1 "2019-04-16T07:18:53Z")
I have 2 different indices within an ElasticSearch cluster with 6 nodes. One index (index A) has 2 primary shards and 2 replicas per giving us 6 total shards. The other index (index B) has 5 primary shards and 1 replica per giving us 10 total shards. When I run the profiler on my query for index A i can see that the query is only being made on 2 different shards(primaries and/or replicas). When I run the profiler on the same query on index B the query is being made on 5 different shards(primaries and/or replicas). It seems that the query is routed to only x shards where x is the number of primary shards designated for that index, but both indices have at least 6 shards for the query to be made on. Is this behavior known or am I missing something? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 16, 2019, 8:10am UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/2 "2019-04-16T08:10:24Z")
Yes, that sounds like the expected behaviour. Each copy of a shard (primary or replica) holds the same data, so there is no point in sending the same search to multiple copies of the same shard. ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 5:02pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/3 "2019-04-16T17:02:01Z")
If I am understanding correctly then a query will only get routed to x number of shards where x is the number of primaries. So even if your schema was 2:5 giving you 10 shards the query would only be sent to 2 shards? Does it make sense that I am seeing much better performance when making the query on index B in the example above with a 5:1 schema vs index A with a 2:2 schema. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 16, 2019, 5:21pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/4 "2019-04-16T17:21:13Z")
Yes, that's correct. However if you send a large number of searches then they will be distributed evenly across all available shards, and that's the case we normally optimise for. ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 5:52pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/5 "2019-04-16T17:52:34Z")
The situation you are describing above is when many queries are ran in parallel correct? In this case ElasticSearch will distribute the queries across all the appropriate shards for the indices being queried. Are you also saying that the single query situation I was describing where the query is sent to multiple shards is being optimized such that the more shards queried yields faster speeds? I guess what I am trying to understand is why the same query runs faster on an index with 5:1 schema vs an index with 2:2 schema. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 16, 2019, 5:58pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/6 "2019-04-16T17:58:42Z")
Do these indices hold exactly the same data? ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 6:02pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/7 "2019-04-16T18:02:38Z")
yes these indices hold the same data, can show screenshots of the query results if that would help/profile api results ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 16, 2019, 6:05pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/8 "2019-04-16T18:05:47Z")
Each query against a shard is single-threaded (queries against different shards run in parallel) which means fewer threads are effectively used for the index that is slower. The shards in the slower index should also be larger which also can impact query latency. Exactly what the difference is will depend on your data and queries. More shards can therefore be faster if you have few concurrent queries but this can change as the number of concurrent queries increase. ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 6:08pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/9 "2019-04-16T18:08:07Z")
It makes sense that the queries run on different shards run in parallel but why does this make the query being routed to 5 shards faster if each thread is doing the exact same thing? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 16, 2019, 6:09pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/10 "2019-04-16T18:09:22Z")
You have 5 threads processing a smaller piece of data each compared to the 2 threads that need to processing considerably more data. It is a simplification but explains the difference. ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 6:11pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/11 "2019-04-16T18:11:51Z")
Does this mean that the work for a single query would be split amongst the 5 threads? I was not aware of this behavior. This is not the same situation as what DavidTurner was describing above where he said a large number of searches will be evenly distributed across all available shards, just to clarify ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 16, 2019, 6:16pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/12 "2019-04-16T18:16:20Z")
> [@arvinds94](#): > > Does this mean that the work for a single query would be split amongst the 5 threads? Yes, a thread per shard is used and if there are not enough threads in the thread pool the jobs are queued up. > [@arvinds94](#): > > This is not the same situation as what DavidTurner was describing above where he said a large number of searches will be evenly distributed across all available shards, just to clarify What I described happen for every query. If you have multiple concurrent queries the load is distributed across the available shards just as David described, but each is still served by multiple threads assuming they target more than one shard. ---
**Author:** ![arvinds94](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@arvinds94](https://discuss.elastic.co/u/arvinds94)\ **Post date:** [April 16, 2019, 6:22pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/13 "2019-04-16T18:22:17Z")
So to give an example, if we had an query/aggregation being performed on an index with a 5:1 shard schema. Different pieces of the result would be returned by each of the 5 threads used to complete the task? Really appreciate your time and patience in explaining this to me ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 16, 2019, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/14 "2019-04-16T18:27:19Z")
Yes, the results from the 5 shards would be returned to the coordinating thread which computes the final result. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 14, 2019, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-number-of-shards-queried/177011/15 "2019-05-14T18:27:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.