# Explain API - All these statistics are per shard not per index **URL:** \ **Category:** Elasticsearch\ **Created:** [April 28, 2020, 1:51pm UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187 "2020-04-28T13:51:56Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 28, 2020, 1:51pm UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/1 "2020-04-28T13:51:57Z")
I am going through this thread > [@What does “docCount” and "docFreq" mean in the Explain API?](https://discuss.elastic.co/t/what-does-doccount-and-docfreq-mean-in-the-explain-api/163850): > > Here's the sample of mapping, register, and search query. mapping curl -X PUT "es:9200/english1" -H 'Content-Type: application/json' -d' { "mappings": { "\_doc": { "properties": { "header" : { "type" : "text" }, "body" : { "type" : "text" } } } } } ' register curl -X PUT "es:9200/english1/\_doc/1?refresh" -H 'Content-Type: application/json' -d' { "header": "something special", "body": "I am John" } ' curl -X PUT "es… Somewhere it is said > All these statistics are per shard not per index I hope this statement is still valid , and i confirmed this behavior too .. What was the reasoning behind this to make the statistics per shard ( not per index) which defeat the purpose of distributed systems as when it supposed to be calculating the relevance in global by looking across all shards ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 28, 2020, 10:03pm UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/2 "2020-04-28T22:03:50Z")
Relevance in Elasticsearch is calculated per shard, not per index. ---
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 29, 2020, 2:21am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/3 "2020-04-29T02:21:58Z")
Thanks a lot, I also mentioned the same. Would be interested know what was the reasoning behind that instead of calculating it per index ? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 29, 2020, 2:46am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/4 "2020-04-29T02:46:34Z")
Because an index is not necessarily on the same node, which means you would need to ship _all_ the documents to a single node to relatively score them. ---
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 29, 2020, 3:10am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/5 "2020-04-29T03:10:20Z")
In practise(considering realtime scenarios and distributed nature of elastic earch), the documents would spread across the nodes - i guess it is the whole purpose of distributed systems like ES. However, the following statement might defeat the whole purpose of distributed nature > you would need to ship _all_ the documents to a single node to relatively score them. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 29, 2020, 3:20am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/6 "2020-04-29T03:20:38Z")
Exactly 🙂 ---
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 29, 2020, 4:13am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/7 "2020-04-29T04:13:19Z")
Not sure you understand my question - what is the reasoning behind doing it per shard ? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 29, 2020, 4:32am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/8 "2020-04-29T04:32:24Z")
I think we're going in circles here 😉 As Elasticsearch is a distributed system, you cannot guarantee that all shards of a given index will be on the same node when a query is processed. If you wanted to do it on an index level, you would need to ship all data from the index to a single node to calculate the relevance. So you either have a single monolithic system to have index level scoring, or a distributed one with shard level scoring. There's costs and benefits to both. ---
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 29, 2020, 11:01am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/9 "2020-04-29T11:01:48Z")
ok, you mean it is more of operational complexity to do global relevance scoring across all shards. So - No other reason , not to do global relevance scoring 🙂 ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 29, 2020, 11:19am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/10 "2020-04-29T11:19:16Z")
Have a look at the different [search types available](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-search.html), especially dfs\_query\_then\_fetch. ---
**Author:** ![nages](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nages/32/46943_2.png) [@nages](https://discuss.elastic.co/u/nages)\ **Post date:** [April 29, 2020, 11:23am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/11 "2020-04-29T11:23:34Z")
thanks for checking , is it any thing different than what @warkolm([https://discuss.elastic.co/u/warkolm](https://discuss.elastic.co/u/warkolm)) referring to ? #justasking ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 27, 2020, 11:23am UTC](https://discuss.elastic.co/t/explain-api-all-these-statistics-are-per-shard-not-per-index/230187/12 "2020-05-27T11:23:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.