# Scoring - queryNorm differs for documents during one query

**URL:** <https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295>\
**Category:** Elasticsearch\
**Created:** [April 17, 2015, 1:00pm UTC](https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295 "2015-04-17T13:00:40Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jakub\_Neubauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakub_neubauer/32/755_2.png) [@Jakub\_Neubauer](https://discuss.elastic.co/u/Jakub_Neubauer)\
**Post date:** [April 17, 2015, 1:00pm UTC](https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295/1 "2015-04-17T13:00:40Z")

</div>

Hi,  
The ES guide states, that when computing score, _"The same query  
normalization factor is applied to every document_" - viz  
[http://www.elastic.co/guide/en/elasticsearch/guide/master/practical-scoring-function.html#query-norm](http://www.elastic.co/guide/en/elasticsearch/guide/master/practical-scoring-function.html#query-norm)  
But when I try this example:

curl -s -XDELETE 'localhost:9200/ttt'

curl -s -XPUT '[http://localhost:9200/ttt/tweet/1?refresh=true](http://localhost:9200/ttt/tweet/1?refresh=true)' -d '{  
"user" : "a b"  
}'  
curl -s -XPUT '[http://localhost:9200/ttt/tweet/2?refresh=true](http://localhost:9200/ttt/tweet/2?refresh=true)' -d '{  
"user" : "b c"  
}'

curl -s -XGET 'localhost:9200/ttt/\_search?explain=trye&format=yaml' -d '  
{  
"query": {  
"match" : { "user" : "a b" }  
}  
}  
'

I got this result - I highlighted the interesting parts:

took: 5  
timed\_out: false  
\_shards:  
total: 5  
successful: 5  
failed: 0  
hits:  
total: 2  
max\_score: 0.2712221  
hits:

- \_shard: 2  
\_node: "\_baxQafwQ0WyAAZpyIv2ow"  
\_index: "ttt"  
\_type: "tweet"  
\_id: "1"  
\_score: 0.2712221  
\_source:  
user: "a b"  
\_explanation:  
value: 0.27122214  
description: "sum of:"  
details:
  - value: 0.13561107  
description: "weight(user:a in 0) [PerFieldSimilarity], result of:"  
details:
    - value: 0.13561107  
description: "score(doc=0,freq=1.0), product of:"  
details:
      - value: 0.70710677  
description: "queryWeight, product of:"  
details:
        - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
        - value: 2.3043842  
description: "queryNorm"

      - value: 0.19178301  
description: "fieldWeight in 0, product of:"  
details:
        - value: 1.0  
description: "tf(freq=1.0), with freq of:"  
details:
          - value: 1.0  
description: "termFreq=1.0"

        - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
        - value: 0.625  
description: "fieldNorm(doc=0)"

  - value: 0.13561107  
description: "weight(user:b in 0) [PerFieldSimilarity], result of:"  
details:
    - value: 0.13561107  
description: "score(doc=0,freq=1.0), product of:"  
details:
      - value: 0.70710677  
description: "queryWeight, product of:"  
details:
        - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
        - value: 2.3043842  
description: "queryNorm"

      - value: 0.19178301  
description: "fieldWeight in 0, product of:"  
details:
        - value: 1.0  
description: "tf(freq=1.0), with freq of:"  
details:
          - value: 1.0  
description: "termFreq=1.0"

        - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
        - value: 0.625  
description: "fieldNorm(doc=0)"

- \_shard: 3  
\_node: "\_baxQafwQ0WyAAZpyIv2ow"  
\_index: "ttt"  
\_type: "tweet"  
\_id: "2"  
\_score: 0.028130025  
\_source:  
user: "b c"  
\_explanation:  
value: 0.028130027  
description: "product of:"  
details:
  - value: 0.056260053  
description: "sum of:"  
details:
    - value: 0.056260053  
description: "weight(user:b in 0) [PerFieldSimilarity], result  
of:"  
details:
      - value: 0.056260053  
description: "score(doc=0,freq=1.0), product of:"  
details:
        - value: 0.29335263  
description: "queryWeight, product of:"  
details:
          - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
          - value: 0.9560043  
description: "queryNorm"

        - value: 0.19178301  
description: "fieldWeight in 0, product of:"  
details:
          - value: 1.0  
description: "tf(freq=1.0), with freq of:"  
details:
            - value: 1.0  
description: "termFreq=1.0"

          - value: 0.30685282  
description: "idf(docFreq=1, maxDocs=1)"
          - value: 0.625  
description: "fieldNorm(doc=0)"

  - value: 0.5  
description: "coord(1/2)"

For the document, where only one term matches, the queryNorm is cca 2.5  
times smaller than at document where both terms match. The result is too  
much penalty for documents matching only one term.  
I can see the same behaviour when using "bool" query with two "should"  
clauses.

Is this a bug? Or what is the explanation of this behaviour? Where can I  
find more info?

Thank you for help

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b3a3953d-9f01-4e78-acf9-44fd95251b81%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b3a3953d-9f01-4e78-acf9-44fd95251b81%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Jakub\_Neubauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakub_neubauer/32/755_2.png) [@Jakub\_Neubauer](https://discuss.elastic.co/u/Jakub_Neubauer)\
**Post date:** [April 17, 2015, 1:09pm UTC](https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295/2 "2015-04-17T13:09:10Z")

</div>

BTW - the reason I'm bothering with this is more complicated. Example in  
the question is already simplified to the core. In my real scenario, I use  
bool query composing more fuzzy queries. Then, the resulting score  
penalizes some documents when only one field matches, which in that case  
has more drastic impact to the user experience. For example exact match in  
one field is far more scored than two fuzzy fields and so on. But the core  
of the problem is the difference of queryNorm between documents which I  
don't understand and cannot find the explanation among the Internet  
resources.

Dne pátek 17. dubna 2015 15:00:40 UTC+2 Jakub Neubauer napsal(a):

> Hi,  
> The ES guide states, that when computing score, _"The same query  
> normalization factor is applied to every document_" - viz  
> [Lucene’s Practical Scoring Function | Elasticsearch: The Definitive Guide [master] | Elastic](http://www.elastic.co/guide/en/elasticsearch/guide/master/practical-scoring-function.html#query-norm)  
> But when I try this example:
> 
> curl -s -XDELETE 'localhost:9200/ttt'
> 
> curl -s -XPUT '[http://localhost:9200/ttt/tweet/1?refresh=true](http://localhost:9200/ttt/tweet/1?refresh=true)' -d '{  
> "user" : "a b"  
> }'  
> curl -s -XPUT '[http://localhost:9200/ttt/tweet/2?refresh=true](http://localhost:9200/ttt/tweet/2?refresh=true)' -d '{  
> "user" : "b c"  
> }'
> 
> curl -s -XGET 'localhost:9200/ttt/\_search?explain=trye&format=yaml' -d '  
> {  
> "query": {  
> "match" : { "user" : "a b" }  
> }  
> }  
> '
> 
> I got this result - I highlighted the interesting parts:
> 
> took: 5  
> timed\_out: false  
> \_shards:  
> total: 5  
> successful: 5  
> failed: 0  
> hits:  
> total: 2  
> max\_score: 0.2712221  
> hits:
> 
> - \_shard: 2  
> \_node: "\_baxQafwQ0WyAAZpyIv2ow"  
> \_index: "ttt"  
> \_type: "tweet"  
> \_id: "1"  
> \_score: 0.2712221  
> \_source:  
> user: "a b"  
> \_explanation:  
> value: 0.27122214  
> description: "sum of:"  
> details:
> - value: 0.13561107  
> description: "weight(user:a in 0) [PerFieldSimilarity], result of:"  
> details:
> - value: 0.13561107  
> description: "score(doc=0,freq=1.0), product of:"  
> details:
> - value: 0.70710677  
> description: "queryWeight, product of:"  
> details:
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 2.3043842  
> description: "queryNorm"
> 
> - value: 0.19178301  
> description: "fieldWeight in 0, product of:"  
> details:
> - value: 1.0  
> description: "tf(freq=1.0), with freq of:"  
> details:
> - value: 1.0  
> description: "termFreq=1.0"
> 
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 0.625  
> description: "fieldNorm(doc=0)"
> 
> - value: 0.13561107  
> description: "weight(user:b in 0) [PerFieldSimilarity], result of:"  
> details:
> - value: 0.13561107  
> description: "score(doc=0,freq=1.0), product of:"  
> details:
> - value: 0.70710677  
> description: "queryWeight, product of:"  
> details:
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 2.3043842  
> description: "queryNorm"
> 
> - value: 0.19178301  
> description: "fieldWeight in 0, product of:"  
> details:
> - value: 1.0  
> description: "tf(freq=1.0), with freq of:"  
> details:
> - value: 1.0  
> description: "termFreq=1.0"
> 
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 0.625  
> description: "fieldNorm(doc=0)"
> 
> - \_shard: 3  
> \_node: "\_baxQafwQ0WyAAZpyIv2ow"  
> \_index: "ttt"  
> \_type: "tweet"  
> \_id: "2"  
> \_score: 0.028130025  
> \_source:  
> user: "b c"  
> \_explanation:  
> value: 0.028130027  
> description: "product of:"  
> details:
> - value: 0.056260053  
> description: "sum of:"  
> details:
> - value: 0.056260053  
> description: "weight(user:b in 0) [PerFieldSimilarity], result  
> of:"  
> details:
> - value: 0.056260053  
> description: "score(doc=0,freq=1.0), product of:"  
> details:
> - value: 0.29335263  
> description: "queryWeight, product of:"  
> details:
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 0.9560043  
> description: "queryNorm"
> 
> - value: 0.19178301  
> description: "fieldWeight in 0, product of:"  
> details:
> - value: 1.0  
> description: "tf(freq=1.0), with freq of:"  
> details:
> - value: 1.0  
> description: "termFreq=1.0"
> 
> - value: 0.30685282  
> description: "idf(docFreq=1, maxDocs=1)"
> - value: 0.625  
> description: "fieldNorm(doc=0)"
> 
> - value: 0.5  
> description: "coord(1/2)"
> 
> For the document, where only one term matches, the queryNorm is cca 2.5  
> times smaller than at document where both terms match. The result is too  
> much penalty for documents matching only one term.  
> I can see the same behaviour when using "bool" query with two "should"  
> clauses.
> 
> Is this a bug? Or what is the explanation of this behaviour? Where can I  
> find more info?
> 
> Thank you for help

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/330119ef-9133-4226-89b8-8e28a843015f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/330119ef-9133-4226-89b8-8e28a843015f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Jakub\_Neubauer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakub_neubauer/32/755_2.png) [@Jakub\_Neubauer](https://discuss.elastic.co/u/Jakub_Neubauer)\
**Post date:** [April 21, 2015, 5:46pm UTC](https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295/3 "2015-04-21T17:46:23Z")

</div>

Just some thoughts: As the queryNorm is calculated from terms frequencies -  
it seems to me, that it is calculated from only those terms of the query  
that somehow "matched" the document in some clause. So in our example, for  
first document terms "a" and "b" were used to calculate queryNorm, but for  
the second document only term "b". But this is not what one would suppose  
from the documentation! I would expect that _all_ query terms would be used  
in calculation, to satisfy the statement that queryNorm is fixed for all  
hits.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/28a667db-e8e1-47b3-a9a2-98cb5a277659%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/28a667db-e8e1-47b3-a9a2-98cb5a277659%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:18am UTC](https://discuss.elastic.co/t/scoring-querynorm-differs-for-documents-during-one-query/23295/4 "2017-07-06T00:18:30Z")

</div>