# Understanding the Explain Query

**URL:** <https://discuss.elastic.co/t/understanding-the-explain-query/8892>\
**Category:** Elasticsearch\
**Created:** [August 30, 2012, 10:39am UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892 "2012-08-30T10:39:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Peter\_Schroder](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/peter_schroder/32/2513_2.png) [@Peter\_Schroder](https://discuss.elastic.co/u/Peter_Schroder)\
**Post date:** [August 30, 2012, 10:39am UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/1 "2012-08-30T10:39:07Z")

</div>

i am trying to debug a scoring issue, but i did not find a proper guide to  
how one would read the explain results.

there is a query that has custom field boosts:

curl -X GET  
"[http://localhost:9200/test-headings/heading/\_search?from=0&load=false&page=1&per\_page=75&raw\_hits=true&size=75&pretty=true](http://localhost:9200/test-headings/heading/_search?from=0&load=false&page=1&per_page=75&raw_hits=true&size=75&pretty=true)"  
-d  
'{"from":0,"size":75,"query":{"query\_string":{"query":"drehteile","fields":["hk\_name\_de^10","hk\_name\_split\_de^4","hk\_recombined\_de","hk\_manual\_de","hk\_manual\_final\_de^0.1","hk\_adjectiv\_de","hk\_singplur\_de","hk\_synonym\_de","hk\_replaced\_de","hk\_adjectiv\_enh\_de","hk\_sea\_enh\_de","hk\_click\_keywords\_de^4.5"],"use\_dis\_max":false}},"filter":{"numeric\_range":{"company\_counts.DE.DE":{"gt":0}}}}'

i thought that the results would be sorted according to the _^X_ modifieres  
like:

1. hk\_name\_de =\> 10
2. hk\_click\_keywords\_de =\> 4.5
3. hk\_name\_split\_de =\> 4
4. rest\_de
5. hk\_manual\_final\_de =\> 0.1

unfortunately the results are not ordered that way. i tried playing around  
with dismax settings, omit\_norms etc, which had no effect on what i was  
doing. my test-data just has one keyword, so the number or length should  
not matter anyways.

the explain-query that i get has the following part:

```
{
  "_shard" : 3,
  "_node" : "dHiSD_vaR46-UaWAjI71JQ",
  "_index" : "test-headings",
  "_type" : "heading",
  "_id" : "95",
  "_score" : 0.049284987, "_source" : {"company_counts":{"DE":{"DE":100,

```

"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,"CH":30}},  
"hk\_name\_de":[],"hk\_name\_split\_de":[],"hk\_recombined\_de":[],"hk\_manual\_de":[  
"drehteile"],"hk\_manual\_final\_de":[],"hk\_adjectiv\_de":[],"hk\_singplur\_de"  
:[],"hk\_synonym\_de":[],"hk\_replaced\_de":[],"hk\_adjectiv\_enh\_de":[],  
"hk\_sea\_enh\_de":[],"hk\_name\_en":[],"hk\_name\_split\_en":[],"hk\_recombined\_en"  
:[],"hk\_manual\_en":[],"hk\_manual\_final\_en":[],"hk\_adjectiv\_en":[],  
"hk\_singplur\_en":[],"hk\_synonym\_en":[],"hk\_replaced\_en":[],  
"hk\_adjectiv\_enh\_en":[],"hk\_sea\_enh\_en":[],"hk\_name\_fr":[],  
"hk\_name\_split\_fr":[],"hk\_recombined\_fr":[],"hk\_manual\_fr":[],  
"hk\_manual\_final\_fr":[],"hk\_adjectiv\_fr":[],"hk\_singplur\_fr":[],  
"hk\_synonym\_fr":[],"hk\_replaced\_fr":[],"hk\_adjectiv\_enh\_fr":[],  
"hk\_sea\_enh\_fr":[],"hk\_name\_nl":[],"hk\_name\_split\_nl":[],"hk\_recombined\_nl"  
:[],"hk\_manual\_nl":[],"hk\_manual\_final\_nl":[],"hk\_adjectiv\_nl":[],  
"hk\_singplur\_nl":[],"hk\_synonym\_nl":[],"hk\_replaced\_nl":[],  
"hk\_adjectiv\_enh\_nl":[],"hk\_sea\_enh\_nl":[],"name\_de":"only\_manual\_drehteile"  
,"id":"95"},  
"highlight" : {  
"hk\_manual\_de" : ["_drehteile_"]  
},  
"\_explanation" : {  
"value" : 0.049284987,  
"description" : "sum of:",  
"details" : [ {  
"value" : 0.049284987,  
"description" : "weight(hk\_manual\_de:drehteile in 1), product of:"  
,  
"details" : [ {  
"value" : 0.049284987,  
"description" : "queryWeight(hk\_manual\_de:drehteile), product  
of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "idf(docFreq=1, maxDocs=2)"  
}, {  
"value" : 0.049284987,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 1.0,  
"description" : "fieldWeight(hk\_manual\_de:drehteile in 1),  
product of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(hk\_manual\_de:drehteile)=1)"  
}, {  
"value" : 1.0,  
"description" : "idf(docFreq=1, maxDocs=2)"  
}, {  
"value" : 1.0,  
"description" : "fieldNorm(field=hk\_manual\_de, doc=1)"  
} ]  
} ]  
} ]  
}  
}, {  
"\_shard" : 2,  
"\_node" : "dHiSD\_vaR46-UaWAjI71JQ",  
"\_index" : "test-headings",  
"\_type" : "heading",  
"\_id" : "94",  
"\_score" : 0.03306274, "\_source" : {"company\_counts":{"DE":{"DE":100,  
"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,"CH":30}},  
"hk\_name\_de":[],"hk\_name\_split\_de":["drehteile"],"hk\_recombined\_de":[],  
"hk\_manual\_de":[],"hk\_manual\_final\_de":[],"hk\_adjectiv\_de":[],  
"hk\_singplur\_de":[],"hk\_synonym\_de":[],"hk\_replaced\_de":[],  
"hk\_adjectiv\_enh\_de":[],"hk\_sea\_enh\_de":[],"hk\_name\_en":[],  
"hk\_name\_split\_en":[],"hk\_recombined\_en":[],"hk\_manual\_en":[],  
"hk\_manual\_final\_en":[],"hk\_adjectiv\_en":[],"hk\_singplur\_en":[],  
"hk\_synonym\_en":[],"hk\_replaced\_en":[],"hk\_adjectiv\_enh\_en":[],  
"hk\_sea\_enh\_en":[],"hk\_name\_fr":[],"hk\_name\_split\_fr":[],"hk\_recombined\_fr"  
:[],"hk\_manual\_fr":[],"hk\_manual\_final\_fr":[],"hk\_adjectiv\_fr":[],  
"hk\_singplur\_fr":[],"hk\_synonym\_fr":[],"hk\_replaced\_fr":[],  
"hk\_adjectiv\_enh\_fr":[],"hk\_sea\_enh\_fr":[],"hk\_name\_nl":[],  
"hk\_name\_split\_nl":[],"hk\_recombined\_nl":[],"hk\_manual\_nl":[],  
"hk\_manual\_final\_nl":[],"hk\_adjectiv\_nl":[],"hk\_singplur\_nl":[],  
"hk\_synonym\_nl":[],"hk\_replaced\_nl":[],"hk\_adjectiv\_enh\_nl":[],  
"hk\_sea\_enh\_nl":[],"name\_de":"only\_split\_drehteile","id":"94"},  
"highlight" : {  
"hk\_name\_split\_de" : ["_drehteile_"]  
},  
"\_explanation" : {  
"value" : 0.03306274,  
"description" : "sum of:",  
"details" : [ {  
"value" : 0.03306274,  
"description" : "weight(hk\_name\_split\_de:drehteile^4.0 in 0),  
product of:",  
"details" : [ {  
"value" : 0.10774788,  
"description" : "queryWeight(hk\_name\_split\_de:drehteile^4.0),  
product of:",  
"details" : [ {  
"value" : 4.0,  
"description" : "boost"  
}, {  
"value" : 0.30685282,  
"description" : "idf(docFreq=1, maxDocs=1)"  
}, {  
"value" : 0.08778466,  
"description" : "queryNorm"  
} ]  
}, {  
"value" : 0.30685282,  
"description" : "fieldWeight(hk\_name\_split\_de:drehteile in 0),  
product of:",  
"details" : [ {  
"value" : 1.0,  
"description" : "tf(termFreq(hk\_name\_split\_de:drehteile)=1)"  
}, {  
"value" : 0.30685282,  
"description" : "idf(docFreq=1, maxDocs=1)"  
}, {  
"value" : 1.0,  
"description" : "fieldNorm(field=hk\_name\_split\_de, doc=0)"  
} ]  
} ]  
} ]  
}  
}

i was expecting hk\_name\_split\_de to be scored higher than hk\_manual\_de:

weight(hk\_manual\_de:drehteile in 1) vs. weight(hk\_name\_split\_de:drehteile^4.0  
in 0)

any pointers why this is not working the way i was expecting?

--

---

<div class="post-metadata">

**Author:** ![Robert\_Moszczynski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robert_moszczynski/32/2727_2.png) [@Robert\_Moszczynski](https://discuss.elastic.co/u/Robert_Moszczynski)\
**Post date:** [September 3, 2012, 8:09pm UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/2 "2012-09-03T20:09:20Z")

</div>

Hi Peter,

I don't think this is an issue. You have to look at your inverted document frequency factor.

An index is not a database and the boosting is not the only factor to determine the scoring between the query and the document. The ^X syntax is used to set a dynamic boosting to a field in query time. Additionally you can use term boosting and document boosting. The boosting values you can find in the mapping are multiplied with other boostings, also the boosting in query time.

Lucene uses a scoring formula to compute the scoring value. This is a function for a query you specified and a index document. You can read about the default similarity here:

[http://lucene.apache.org/core/3\_6\_0/api/all/org/apache/lucene/search/Similarity.html](http://lucene.apache.org/core/3_6_0/api/all/org/apache/lucene/search/Similarity.html)

Let us analyze your explanation structure. The score value is a sum of weights for every query term multiplied with the coordination factor (here omitted 1/1=1). You used only one search term 'drehteile' therefore, only one summand. The weight for a term is a product of a queryWeight and fieldWeight for the term. Since the cooridination factor was omitted there are two main factors in the queryWeight. Inverted document frequency describes the rarity of the term in the index corpus - rare terms are scored higher (logarithmic behaviour). The second factor is the queryNorm which is a normalizing factor in the query time. This includes also structural differences (field sets in query and document). In the fieldWeight factor you can see the heart of Lucene scoring, the tf-idf component. The term frequency multiplied by the inverted document frequency describes the main scoring behaviour. Term frequency is the number of times the term appears in the field.

Of course this is not a complete guide through the Lucene Similarty, but shows you the idea behind the similarity formula.

Dismax you can use when you need run more queries and control the computation of the scores of the queries into one value. With tie\_breaker you can controll the level of scores should be included in the calculation.

With omit\_norms (indexing time - you should put this into your mapping) you can turn off for example the field length. Lucene uses the field length (number of terms in field) by default to calculate a additional relevance factor.

Each of the settings can be helpful or harmful depending on how you have modeled your relevance.

Robert

---

<div class="post-metadata">

**Author:** ![Woody\_Peterson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/woody_peterson/32/830_2.png) [@Woody\_Peterson](https://discuss.elastic.co/u/Woody_Peterson)\
**Post date:** [September 14, 2012, 6:36pm UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/3 "2012-09-14T18:36:44Z")

</div>

I recently ran into what looks to be a similar issue while testing. As  
Robert pointed out, your inverse document frequency numbers are messing  
with your results, BUT more specifically it's due to your low document  
numbers interacting with what is probably a default shard number of 5 in  
your setup. Since each shard is its own lucene instance, if you have say 6  
documents in elasticsearch, one shard (lucene instance) will have 2 total  
documents, while the other shards will have 1 total. Thus, when the  
2-document shard gets a hit on 1 document for a term, it will think it is  
relatively more rare of a term than the 1-document shards where the term  
appears in 100% of the documents (1 out of 1).

The solution to eliminating huge discrepancies in idf for low document  
numbers is to set number\_of\_shards to 1 (possibly number\_of\_replicas to 1  
also, unsure how replicas come in to play here).

On Thursday, August 30, 2012 3:39:07 AM UTC-7, Peter Schröder wrote:

> i am trying to debug a scoring issue, but i did not find a proper guide to  
> how one would read the explain results.
> 
> there is a query that has custom field boosts:
> 
> curl -X GET "  
> [http://localhost:9200/test-headings/heading/\_search?from=0&load=false&page=1&per\_page=75&raw\_hits=true&size=75&pretty=true](http://localhost:9200/test-headings/heading/_search?from=0&load=false&page=1&per_page=75&raw_hits=true&size=75&pretty=true)  
> " -d  
> '{"from":0,"size":75,"query":{"query\_string":{"query":"drehteile","fields":["hk\_name\_de^10","hk\_name\_split\_de^4","hk\_recombined\_de","hk\_manual\_de","hk\_manual\_final\_de^0.1","hk\_adjectiv\_de","hk\_singplur\_de","hk\_synonym\_de","hk\_replaced\_de","hk\_adjectiv\_enh\_de","hk\_sea\_enh\_de","hk\_click\_keywords\_de^4.5"],"use\_dis\_max":false}},"filter":{"numeric\_range":{"  
> company\_counts.DE.DE":{"gt":0}}}}'
> 
> i thought that the results would be sorted according to the _^X_modifieres like:
> 
> 1. hk\_name\_de =\> 10
> 2. hk\_click\_keywords\_de =\> 4.5
> 3. hk\_name\_split\_de =\> 4
> 4. rest\_de
> 5. hk\_manual\_final\_de =\> 0.1
> 
> unfortunately the results are not ordered that way. i tried playing around  
> with dismax settings, omit\_norms etc, which had no effect on what i was  
> doing. my test-data just has one keyword, so the number or length should  
> not matter anyways.
> 
> the explain-query that i get has the following part:
> 
> ```
> {
> "_shard" : 3,
> "_node" : "dHiSD_vaR46-UaWAjI71JQ",
> "_index" : "test-headings",
> "_type" : "heading",
> "_id" : "95",
> "_score" : 0.049284987, "_source" : {"company_counts":{"DE":{"DE":
> 
> ```
> 
> 100,"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,"CH"  
> :30}},"hk\_name\_de":,"hk\_name\_split\_de":,"hk\_recombined\_de":,  
> "hk\_manual\_de":["drehteile"],"hk\_manual\_final\_de":,"hk\_adjectiv\_de":,  
> "hk\_singplur\_de":,"hk\_synonym\_de":,"hk\_replaced\_de":,  
> "hk\_adjectiv\_enh\_de":,"hk\_sea\_enh\_de":,"hk\_name\_en":,  
> "hk\_name\_split\_en":,"hk\_recombined\_en":,"hk\_manual\_en":,  
> "hk\_manual\_final\_en":,"hk\_adjectiv\_en":,"hk\_singplur\_en":,  
> "hk\_synonym\_en":,"hk\_replaced\_en":,"hk\_adjectiv\_enh\_en":,  
> "hk\_sea\_enh\_en":,"hk\_name\_fr":,"hk\_name\_split\_fr":,  
> "hk\_recombined\_fr":,"hk\_manual\_fr":,"hk\_manual\_final\_fr":,  
> "hk\_adjectiv\_fr":,"hk\_singplur\_fr":,"hk\_synonym\_fr":,  
> "hk\_replaced\_fr":,"hk\_adjectiv\_enh\_fr":,"hk\_sea\_enh\_fr":,  
> "hk\_name\_nl":,"hk\_name\_split\_nl":,"hk\_recombined\_nl":,"hk\_manual\_nl"  
> :,"hk\_manual\_final\_nl":,"hk\_adjectiv\_nl":,"hk\_singplur\_nl":,  
> "hk\_synonym\_nl":,"hk\_replaced\_nl":,"hk\_adjectiv\_enh\_nl":,  
> "hk\_sea\_enh\_nl":,"name\_de":"only\_manual\_drehteile","id":"95"},  
> "highlight" : {  
> "hk\_manual\_de" : ["_drehteile_"]  
> },  
> "\_explanation" : {  
> "value" : 0.049284987,  
> "description" : "sum of:",  
> "details" : [ {  
> "value" : 0.049284987,  
> "description" : "weight(hk\_manual\_de:drehteile in 1), product  
> of:",  
> "details" : [ {  
> "value" : 0.049284987,  
> "description" : "queryWeight(hk\_manual\_de:drehteile), product  
> of:",  
> "details" : [ {  
> "value" : 1.0,  
> "description" : "idf(docFreq=1, maxDocs=2)"  
> }, {  
> "value" : 0.049284987,  
> "description" : "queryNorm"  
> } ]  
> }, {  
> "value" : 1.0,  
> "description" : "fieldWeight(hk\_manual\_de:drehteile in 1),  
> product of:",  
> "details" : [ {  
> "value" : 1.0,  
> "description" : "tf(termFreq(hk\_manual\_de:drehteile)=1)"  
> }, {  
> "value" : 1.0,  
> "description" : "idf(docFreq=1, maxDocs=2)"  
> }, {  
> "value" : 1.0,  
> "description" : "fieldNorm(field=hk\_manual\_de, doc=1)"  
> } ]  
> } ]  
> } ]  
> }  
> }, {  
> "\_shard" : 2,  
> "\_node" : "dHiSD\_vaR46-UaWAjI71JQ",  
> "\_index" : "test-headings",  
> "\_type" : "heading",  
> "\_id" : "94",  
> "\_score" : 0.03306274, "\_source" : {"company\_counts":{"DE":{"DE":100  
> ,"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,"CH":30  
> }},"hk\_name\_de":,"hk\_name\_split\_de":["drehteile"],"hk\_recombined\_de":,  
> "hk\_manual\_de":,"hk\_manual\_final\_de":,"hk\_adjectiv\_de":,  
> "hk\_singplur\_de":,"hk\_synonym\_de":,"hk\_replaced\_de":,  
> "hk\_adjectiv\_enh\_de":,"hk\_sea\_enh\_de":,"hk\_name\_en":,  
> "hk\_name\_split\_en":,"hk\_recombined\_en":,"hk\_manual\_en":,  
> "hk\_manual\_final\_en":,"hk\_adjectiv\_en":,"hk\_singplur\_en":,  
> "hk\_synonym\_en":,"hk\_replaced\_en":,"hk\_adjectiv\_enh\_en":,  
> "hk\_sea\_enh\_en":,"hk\_name\_fr":,"hk\_name\_split\_fr":,  
> "hk\_recombined\_fr":,"hk\_manual\_fr":,"hk\_manual\_final\_fr":,  
> "hk\_adjectiv\_fr":,"hk\_singplur\_fr":,"hk\_synonym\_fr":,  
> "hk\_replaced\_fr":,"hk\_adjectiv\_enh\_fr":,"hk\_sea\_enh\_fr":,  
> "hk\_name\_nl":,"hk\_name\_split\_nl":,"hk\_recombined\_nl":,"hk\_manual\_nl"  
> :,"hk\_manual\_final\_nl":,"hk\_adjectiv\_nl":,"hk\_singplur\_nl":,  
> "hk\_synonym\_nl":,"hk\_replaced\_nl":,"hk\_adjectiv\_enh\_nl":,  
> "hk\_sea\_enh\_nl":,"name\_de":"only\_split\_drehteile","id":"94"},  
> "highlight" : {  
> "hk\_name\_split\_de" : ["_drehteile_"]  
> },  
> "\_explanation" : {  
> "value" : 0.03306274,  
> "description" : "sum of:",  
> "details" : [ {  
> "value" : 0.03306274,  
> "description" : "weight(hk\_name\_split\_de:drehteile^4.0 in 0),  
> product of:",  
> "details" : [ {  
> "value" : 0.10774788,  
> "description" : "queryWeight(hk\_name\_split\_de:drehteile^4.0),  
> product of:",  
> "details" : [ {  
> "value" : 4.0,  
> "description" : "boost"  
> }, {  
> "value" : 0.30685282,  
> "description" : "idf(docFreq=1, maxDocs=1)"  
> }, {  
> "value" : 0.08778466,  
> "description" : "queryNorm"  
> } ]  
> }, {  
> "value" : 0.30685282,  
> "description" : "fieldWeight(hk\_name\_split\_de:drehteile in  
> 0), product of:",  
> "details" : [ {  
> "value" : 1.0,  
> "description" : "tf(termFreq(hk\_name\_split\_de:drehteile)=1)"  
> }, {  
> "value" : 0.30685282,  
> "description" : "idf(docFreq=1, maxDocs=1)"  
> }, {  
> "value" : 1.0,  
> "description" : "fieldNorm(field=hk\_name\_split\_de, doc=0)"  
> } ]  
> } ]  
> } ]  
> }  
> }
> 
> i was expecting hk\_name\_split\_de to be scored higher than hk\_manual\_de:
> 
> weight(hk\_manual\_de:drehteile in 1) vs. weight(hk\_name\_split\_de:drehteile^4.0  
> in 0)
> 
> any pointers why this is not working the way i was expecting?

--

---

<div class="post-metadata">

**Author:** ![BillyEm](https://avatars.discourse-cdn.com/v4/letter/b/c4cdca/32.png) [@BillyEm](https://discuss.elastic.co/u/BillyEm)\
**Post date:** [September 15, 2012, 4:03am UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/4 "2012-09-15T04:03:13Z")

</div>

ouch!

On Friday, September 14, 2012 2:36:44 PM UTC-4, Woody Peterson wrote:

> I recently ran into what looks to be a similar issue while testing. As  
> Robert pointed out, your inverse document frequency numbers are messing  
> with your results, BUT more specifically it's due to your low document  
> numbers interacting with what is probably a default shard number of 5 in  
> your setup. Since each shard is its own lucene instance, if you have say 6  
> documents in elasticsearch, one shard (lucene instance) will have 2 total  
> documents, while the other shards will have 1 total. Thus, when the  
> 2-document shard gets a hit on 1 document for a term, it will think it is  
> relatively more rare of a term than the 1-document shards where the term  
> appears in 100% of the documents (1 out of 1).
> 
> The solution to eliminating huge discrepancies in idf for low document  
> numbers is to set number\_of\_shards to 1 (possibly number\_of\_replicas to 1  
> also, unsure how replicas come in to play here).
> 
> On Thursday, August 30, 2012 3:39:07 AM UTC-7, Peter Schröder wrote:
> 
> > i am trying to debug a scoring issue, but i did not find a proper guide  
> > to how one would read the explain results.
> > 
> > there is a query that has custom field boosts:
> > 
> > curl -X GET "  
> > [http://localhost:9200/test-headings/heading/\_search?from=0&load=false&page=1&per\_page=75&raw\_hits=true&size=75&pretty=true](http://localhost:9200/test-headings/heading/_search?from=0&load=false&page=1&per_page=75&raw_hits=true&size=75&pretty=true)  
> > " -d  
> > '{"from":0,"size":75,"query":{"query\_string":{"query":"drehteile","fields":["hk\_name\_de^10","hk\_name\_split\_de^4","hk\_recombined\_de","hk\_manual\_de","hk\_manual\_final\_de^0.1","hk\_adjectiv\_de","hk\_singplur\_de","hk\_synonym\_de","hk\_replaced\_de","hk\_adjectiv\_enh\_de","hk\_sea\_enh\_de","hk\_click\_keywords\_de^4.5"],"use\_dis\_max":false}},"filter":{"numeric\_range":{"  
> > company\_counts.DE.DE":{"gt":0}}}}'
> > 
> > i thought that the results would be sorted according to the _^X_modifieres like:
> > 
> > 1. hk\_name\_de =\> 10
> > 2. hk\_click\_keywords\_de =\> 4.5
> > 3. hk\_name\_split\_de =\> 4
> > 4. rest\_de
> > 5. hk\_manual\_final\_de =\> 0.1
> > 
> > unfortunately the results are not ordered that way. i tried playing  
> > around with dismax settings, omit\_norms etc, which had no effect on what i  
> > was doing. my test-data just has one keyword, so the number or length  
> > should not matter anyways.
> > 
> > the explain-query that i get has the following part:
> > 
> > ```
> > {
> > "_shard" : 3,
> > "_node" : "dHiSD_vaR46-UaWAjI71JQ",
> > "_index" : "test-headings",
> > "_type" : "heading",
> > "_id" : "95",
> > "_score" : 0.049284987, "_source" : {"company_counts":{"DE":{"DE":
> > 
> > ```
> > 
> > 100,"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,  
> > "CH":30}},"hk\_name\_de":,"hk\_name\_split\_de":,"hk\_recombined\_de":,  
> > "hk\_manual\_de":["drehteile"],"hk\_manual\_final\_de":,"hk\_adjectiv\_de":,  
> > "hk\_singplur\_de":,"hk\_synonym\_de":,"hk\_replaced\_de":,  
> > "hk\_adjectiv\_enh\_de":,"hk\_sea\_enh\_de":,"hk\_name\_en":,  
> > "hk\_name\_split\_en":,"hk\_recombined\_en":,"hk\_manual\_en":,  
> > "hk\_manual\_final\_en":,"hk\_adjectiv\_en":,"hk\_singplur\_en":,  
> > "hk\_synonym\_en":,"hk\_replaced\_en":,"hk\_adjectiv\_enh\_en":,  
> > "hk\_sea\_enh\_en":,"hk\_name\_fr":,"hk\_name\_split\_fr":,  
> > "hk\_recombined\_fr":,"hk\_manual\_fr":,"hk\_manual\_final\_fr":,  
> > "hk\_adjectiv\_fr":,"hk\_singplur\_fr":,"hk\_synonym\_fr":,  
> > "hk\_replaced\_fr":,"hk\_adjectiv\_enh\_fr":,"hk\_sea\_enh\_fr":,  
> > "hk\_name\_nl":,"hk\_name\_split\_nl":,"hk\_recombined\_nl":,  
> > "hk\_manual\_nl":,"hk\_manual\_final\_nl":,"hk\_adjectiv\_nl":,  
> > "hk\_singplur\_nl":,"hk\_synonym\_nl":,"hk\_replaced\_nl":,  
> > "hk\_adjectiv\_enh\_nl":,"hk\_sea\_enh\_nl":,"name\_de":  
> > "only\_manual\_drehteile","id":"95"},  
> > "highlight" : {  
> > "hk\_manual\_de" : ["_drehteile_"]  
> > },  
> > "\_explanation" : {  
> > "value" : 0.049284987,  
> > "description" : "sum of:",  
> > "details" : [ {  
> > "value" : 0.049284987,  
> > "description" : "weight(hk\_manual\_de:drehteile in 1), product  
> > of:",  
> > "details" : [ {  
> > "value" : 0.049284987,  
> > "description" : "queryWeight(hk\_manual\_de:drehteile),  
> > product of:",  
> > "details" : [ {  
> > "value" : 1.0,  
> > "description" : "idf(docFreq=1, maxDocs=2)"  
> > }, {  
> > "value" : 0.049284987,  
> > "description" : "queryNorm"  
> > } ]  
> > }, {  
> > "value" : 1.0,  
> > "description" : "fieldWeight(hk\_manual\_de:drehteile in 1),  
> > product of:",  
> > "details" : [ {  
> > "value" : 1.0,  
> > "description" : "tf(termFreq(hk\_manual\_de:drehteile)=1)"  
> > }, {  
> > "value" : 1.0,  
> > "description" : "idf(docFreq=1, maxDocs=2)"  
> > }, {  
> > "value" : 1.0,  
> > "description" : "fieldNorm(field=hk\_manual\_de, doc=1)"  
> > } ]  
> > } ]  
> > } ]  
> > }  
> > }, {  
> > "\_shard" : 2,  
> > "\_node" : "dHiSD\_vaR46-UaWAjI71JQ",  
> > "\_index" : "test-headings",  
> > "\_type" : "heading",  
> > "\_id" : "94",  
> > "\_score" : 0.03306274, "\_source" : {"company\_counts":{"DE":{"DE":  
> > 100,"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT":5,  
> > "CH":30}},"hk\_name\_de":,"hk\_name\_split\_de":["drehteile"],  
> > "hk\_recombined\_de":,"hk\_manual\_de":,"hk\_manual\_final\_de":,  
> > "hk\_adjectiv\_de":,"hk\_singplur\_de":,"hk\_synonym\_de":,  
> > "hk\_replaced\_de":,"hk\_adjectiv\_enh\_de":,"hk\_sea\_enh\_de":,  
> > "hk\_name\_en":,"hk\_name\_split\_en":,"hk\_recombined\_en":,  
> > "hk\_manual\_en":,"hk\_manual\_final\_en":,"hk\_adjectiv\_en":,  
> > "hk\_singplur\_en":,"hk\_synonym\_en":,"hk\_replaced\_en":,  
> > "hk\_adjectiv\_enh\_en":,"hk\_sea\_enh\_en":,"hk\_name\_fr":,  
> > "hk\_name\_split\_fr":,"hk\_recombined\_fr":,"hk\_manual\_fr":,  
> > "hk\_manual\_final\_fr":,"hk\_adjectiv\_fr":,"hk\_singplur\_fr":,  
> > "hk\_synonym\_fr":,"hk\_replaced\_fr":,"hk\_adjectiv\_enh\_fr":,  
> > "hk\_sea\_enh\_fr":,"hk\_name\_nl":,"hk\_name\_split\_nl":,  
> > "hk\_recombined\_nl":,"hk\_manual\_nl":,"hk\_manual\_final\_nl":,  
> > "hk\_adjectiv\_nl":,"hk\_singplur\_nl":,"hk\_synonym\_nl":,  
> > "hk\_replaced\_nl":,"hk\_adjectiv\_enh\_nl":,"hk\_sea\_enh\_nl":,"name\_de":  
> > "only\_split\_drehteile","id":"94"},  
> > "highlight" : {  
> > "hk\_name\_split\_de" : ["_drehteile_"]  
> > },  
> > "\_explanation" : {  
> > "value" : 0.03306274,  
> > "description" : "sum of:",  
> > "details" : [ {  
> > "value" : 0.03306274,  
> > "description" : "weight(hk\_name\_split\_de:drehteile^4.0 in 0),  
> > product of:",  
> > "details" : [ {  
> > "value" : 0.10774788,  
> > "description" : "queryWeight(hk\_name\_split\_de:drehteile^4.0),  
> > product of:",  
> > "details" : [ {  
> > "value" : 4.0,  
> > "description" : "boost"  
> > }, {  
> > "value" : 0.30685282,  
> > "description" : "idf(docFreq=1, maxDocs=1)"  
> > }, {  
> > "value" : 0.08778466,  
> > "description" : "queryNorm"  
> > } ]  
> > }, {  
> > "value" : 0.30685282,  
> > "description" : "fieldWeight(hk\_name\_split\_de:drehteile in  
> > 0), product of:",  
> > "details" : [ {  
> > "value" : 1.0,  
> > "description" :  
> > "tf(termFreq(hk\_name\_split\_de:drehteile)=1)"  
> > }, {  
> > "value" : 0.30685282,  
> > "description" : "idf(docFreq=1, maxDocs=1)"  
> > }, {  
> > "value" : 1.0,  
> > "description" : "fieldNorm(field=hk\_name\_split\_de, doc=0)"  
> > } ]  
> > } ]  
> > } ]  
> > }  
> > }
> > 
> > i was expecting hk\_name\_split\_de to be scored higher than hk\_manual\_de:
> > 
> > weight(hk\_manual\_de:drehteile in 1) vs. weight(hk\_name\_split\_de:drehteile^4.0  
> > in 0)
> > 
> > any pointers why this is not working the way i was expecting?

--

---

<div class="post-metadata">

**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\
**Post date:** [September 15, 2012, 6:37pm UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/5 "2012-09-15T18:37:05Z")

</div>

Have you tried different search types?

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Regards,  
Lukas

On Sat, Sep 15, 2012 at 6:03 AM, BillyEm [wmartinusa@gmail.com](mailto:wmartinusa@gmail.com) wrote:

> ouch!
> 
> On Friday, September 14, 2012 2:36:44 PM UTC-4, Woody Peterson wrote:
> 
> > I recently ran into what looks to be a similar issue while testing. As  
> > Robert pointed out, your inverse document frequency numbers are messing  
> > with your results, BUT more specifically it's due to your low document  
> > numbers interacting with what is probably a default shard number of 5 in  
> > your setup. Since each shard is its own lucene instance, if you have say 6  
> > documents in elasticsearch, one shard (lucene instance) will have 2 total  
> > documents, while the other shards will have 1 total. Thus, when the  
> > 2-document shard gets a hit on 1 document for a term, it will think it is  
> > relatively more rare of a term than the 1-document shards where the term  
> > appears in 100% of the documents (1 out of 1).
> > 
> > The solution to eliminating huge discrepancies in idf for low document  
> > numbers is to set number\_of\_shards to 1 (possibly number\_of\_replicas to 1  
> > also, unsure how replicas come in to play here).
> > 
> > On Thursday, August 30, 2012 3:39:07 AM UTC-7, Peter Schröder wrote:
> > 
> > > i am trying to debug a scoring issue, but i did not find a proper guide  
> > > to how one would read the explain results.
> > > 
> > > there is a query that has custom field boosts:
> > > 
> > > curl -X GET "[http://localhost:9200/test-\*\*headings/heading/\_search?from=](http://localhost:9200/test-**headings/heading/_search?from=)  
> > > \*\*0&load=false&page=1&per\_page=**75&raw\_hits=true&size=75&pretty=true[http://localhost:9200/test-headings/heading/\_search?from=0&load=false&page=1&per\_page=75&raw\_hits=true&size=75&pretty=true](http://localhost:9200/test-headings/heading/_search?from=0&load=false&page=1&per_page=75&raw_hits=true&size=75&pretty=true)  
> > > " -d '{"from":0,"size":75,"query":{"query\_string":{"query":"**  
> > > drehteile","fields":["hk\_name\_ **de^10","hk\_name\_split\_de^4","**  
> > > hk\_recombined\_de","hk\_manual\_ **de","hk\_manual\_final\_de^0.1","**  
> > > hk\_adjectiv\_de","hk\_singplur\_ **de","hk\_synonym\_de","hk\_**  
> > > replaced\_de","hk\_adjectiv\_enh\_ **de","hk\_sea\_enh\_de","hk\_click\_**  
> > > keywords\_de^4.5"],"use\_dis\_\*\*max":false}},"filter":{"\*\*numeric\_range":{"  
> > > company\_\*\*[counts.DE.DE](http://counts.DE.DE) [http://company\_counts.DE.DE](http://company_counts.DE.DE)":{"gt":0}}}}'
> > > 
> > > i thought that the results would be sorted according to the _^X_modifieres like:
> > > 
> > > 1. hk\_name\_de =\> 10
> > > 2. hk\_click\_keywords\_de =\> 4.5
> > > 3. hk\_name\_split\_de =\> 4
> > > 4. rest\_de
> > > 5. hk\_manual\_final\_de =\> 0.1
> > > 
> > > unfortunately the results are not ordered that way. i tried playing  
> > > around with dismax settings, omit\_norms etc, which had no effect on what i  
> > > was doing. my test-data just has one keyword, so the number or length  
> > > should not matter anyways.
> > > 
> > > the explain-query that i get has the following part:
> > > 
> > > ```
> > > {
> > > "_shard" : 3,
> > > "_node" : "dHiSD_vaR46-UaWAjI71JQ",
> > > "_index" : "test-headings",
> > > "_type" : "heading",
> > > "_id" : "95",
> > > "_score" : 0.049284987, "_source" : {"company_counts":{"DE":{"DE":
> > > 
> > > ```
> > > 
> > > **100,"AT":10,"CH":1},"AT":{"DE"** :2,"AT":50,"CH":10},"CH":{"DE"\*\*:1,  
> > > "AT":5,"CH":30}},"hk\_name\_**de":[],"hk\_name\_split\_de":[],"**  
> > > hk\_recombined\_de":,"hk\_**manual\_de":["drehteile"],"hk\_**  
> > > manual\_final\_de":,"hk\_\*\*adjectiv\_de":,"hk\_singplur\_\*\*de":,  
> > > "hk\_synonym\_de":,"hk\_\*\*replaced\_de":,"hk\_adjectiv\_**enh\_de":[],  
> > > "hk\_sea\_enh\_de":[],**"hk\_name\_en":,"hk\_name\_\*\*split\_en":,  
> > > "hk\_recombined\_\*\*en":,"hk\_manual\_en":,"hk\_\*\*manual\_final\_en":,"hk\_  
> > > \*\*adjectiv\_en":,"hk\_singplur\_**en":[],"hk\_synonym\_en":[],"hk\_**  
> > > replaced\_en":,"hk\_adjectiv\_**enh\_en":[],"hk\_sea\_enh\_en":[],**  
> > > "hk\_name\_fr":,"hk\_name\_\*\*split\_fr":,"hk\_recombined\_\*\*fr":,  
> > > "hk\_manual\_fr":,"hk\_\*\*manual\_final\_fr":,"hk\_\*\*adjectiv\_fr":,  
> > > "hk\_singplur\_\*\*fr":,"hk\_synonym\_fr":,"hk\_\*\*replaced\_fr":,  
> > > "hk\_adjectiv\_**enh\_fr":[],"hk\_sea\_enh\_fr":[],**"hk\_name\_nl":,"hk\_name\_  
> > > \*\*split\_nl":,"hk\_recombined\_**nl":[],"hk\_manual\_nl":[],"hk\_**  
> > > manual\_final\_nl":,"hk\_\*\*adjectiv\_nl":,"hk\_singplur\_\*\*nl":,  
> > > "hk\_synonym\_nl":,"hk\_\*\*replaced\_nl":,"hk\_adjectiv\_**enh\_nl":[],  
> > > "hk\_sea\_enh\_nl":[],**"name\_de":"only\_manual\_\*\*drehteile","id":"95"},  
> > > "highlight" : {  
> > > "hk\_manual\_de" : ["_drehteile_"]  
> > > },  
> > > "_explanation" : {  
> > > "value" : 0.049284987,  
> > > "description" : "sum of:",  
> > > "details" : [ {  
> > > "value" : 0.049284987,  
> > > "description" : "weight(hk\_manual\_de:drehteile in 1), product  
> > > of:",  
> > > "details" : [ {  
> > > "value" : 0.049284987,  
> > > "description" : "queryWeight(hk\_manual\_de:\*drehteile),  
> > > product of:",  
> > > "details" : [ {  
> > > "value" : 1.0,  
> > > "description" : "idf(docFreq=1, maxDocs=2)"  
> > > }, {  
> > > "value" : 0.049284987,  
> > > "description" : "queryNorm"  
> > > } ]  
> > > }, {  
> > > "value" : 1.0,  
> > > "description" : "fieldWeight(hk\_manual\_de:drehteile in  
> > > 1), product of:",  
> > > "details" : [ {  
> > > "value" : 1.0,  
> > > "description" : "tf(termFreq(hk\_manual\_de:drehteile)=1)"  
> > > }, {  
> > > "value" : 1.0,  
> > > "description" : "idf(docFreq=1, maxDocs=2)"  
> > > }, {  
> > > "value" : 1.0,  
> > > "description" : "fieldNorm(field=hk\_manual\_de, doc=1)"  
> > > } ]  
> > > } ]  
> > > } ]  
> > > }  
> > > }, {  
> > > "\_shard" : 2,  
> > > "\_node" : "dHiSD\_vaR46-UaWAjI71JQ",  
> > > "\_index" : "test-headings",  
> > > "\_type" : "heading",  
> > > "\_id" : "94",  
> > > "\_score" : 0.03306274, "\_source" : {"company\_counts":{"DE":{"DE":  
> > > \*100,"AT":10,"CH":1},"AT":{"DE":2,"AT":50,"CH":10},"CH":{"DE":1,"AT"  
> > > :5,"CH":30}},"hk\_name_\*\*de":,"hk\_name\_split\_de":["**drehteile"],  
> > > "hk\_recombined\_de"**:,"hk\_manual\_de":,"hk\_\*\*manual\_final\_de":,"hk\_  
> > > \*\*adjectiv\_de":,"hk\_singplur\_**de":[],"hk\_synonym\_de":[],"hk\_**  
> > > replaced\_de":,"hk\_adjectiv\_**enh\_de":[],"hk\_sea\_enh\_de":[],**  
> > > "hk\_name\_en":,"hk\_name\_\*\*split\_en":,"hk\_recombined\_\*\*en":,  
> > > "hk\_manual\_en":,"hk\_\*\*manual\_final\_en":,"hk\_\*\*adjectiv\_en":,  
> > > "hk\_singplur\_\*\*en":,"hk\_synonym\_en":,"hk\_\*\*replaced\_en":,  
> > > "hk\_adjectiv\_**enh\_en":[],"hk\_sea\_enh\_en":[],**"hk\_name\_fr":,"hk\_name\_  
> > > \*\*split\_fr":,"hk\_recombined\_**fr":[],"hk\_manual\_fr":[],"hk\_**  
> > > manual\_final\_fr":,"hk\_\*\*adjectiv\_fr":,"hk\_singplur\_\*\*fr":,  
> > > "hk\_synonym\_fr":,"hk\_\*\*replaced\_fr":,"hk\_adjectiv\_**enh\_fr":[],  
> > > "hk\_sea\_enh\_fr":[],**"hk\_name\_nl":,"hk\_name\_\*\*split\_nl":,  
> > > "hk\_recombined\_\*\*nl":,"hk\_manual\_nl":,"hk\_\*\*manual\_final\_nl":,"hk\_  
> > > \*\*adjectiv\_nl":,"hk\_singplur\_**nl":[],"hk\_synonym\_nl":[],"hk\_**  
> > > replaced\_nl":,"hk\_adjectiv\_**enh\_nl":[],"hk\_sea\_enh\_nl":[],**"name\_de"  
> > > :"only\_split\_\*\*drehteile","id":"94"},  
> > > "highlight" : {  
> > > "hk\_name\_split\_de" : ["_drehteile_"]  
> > > },  
> > > "_explanation" : {  
> > > "value" : 0.03306274,  
> > > "description" : "sum of:",  
> > > "details" : [ {  
> > > "value" : 0.03306274,  
> > > "description" : "weight(hk\_name\_split\_de:\*\*drehteile^4.0 in  
> > > 0), product of:",  
> > > "details" : [ {  
> > > "value" : 0.10774788,  
> > > "description" : "queryWeight(hk\_name\_split\_de:\*\*drehteile^4.0),  
> > > product of:",  
> > > "details" : [ {  
> > > "value" : 4.0,  
> > > "description" : "boost"  
> > > }, {  
> > > "value" : 0.30685282,  
> > > "description" : "idf(docFreq=1, maxDocs=1)"  
> > > }, {  
> > > "value" : 0.08778466,  
> > > "description" : "queryNorm"  
> > > } ]  
> > > }, {  
> > > "value" : 0.30685282,  
> > > "description" : "fieldWeight(hk\_name\_split\_de:**drehteile  
> > > in 0), product of:",  
> > > "details" : [ {  
> > > "value" : 1.0,  
> > > "description" : "tf(termFreq(hk\_name\_split\_de:**  
> > > drehteile)=1)"  
> > > }, {  
> > > "value" : 0.30685282,  
> > > "description" : "idf(docFreq=1, maxDocs=1)"  
> > > }, {  
> > > "value" : 1.0,  
> > > "description" : "fieldNorm(field=hk\_name_\*\*split\_de,  
> > > doc=0)"  
> > > } ]  
> > > } ]  
> > > } ]  
> > > }  
> > > }
> > > 
> > > i was expecting hk\_name\_split\_de to be scored higher than hk\_manual\_de:
> > > 
> > > weight(hk\_manual\_de:drehteile in 1) vs. weight(hk\_name\_split\_\*\*de:drehteile^4.0  
> > > in 0)
> > > 
> > > any pointers why this is not working the way i was expecting?
> > > 
> > > --

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:12am UTC](https://discuss.elastic.co/t/understanding-the-explain-query/8892/6 "2017-07-06T03:12:40Z")

</div>