# Surprising scoring when using multi\_match's cross\_field **URL:** \ **Category:** Elasticsearch\ **Created:** [June 27, 2014, 1:02pm UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371 "2014-06-27T13:02:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Christoph\_Lingg](https://avatars.discourse-cdn.com/v4/letter/c/5f9b8f/32.png) [@Christoph\_Lingg](https://discuss.elastic.co/u/Christoph_Lingg)\ **Post date:** [June 27, 2014, 1:02pm UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/1 "2014-06-27T13:02:14Z")
Hello! I am using the multi\_match's cross\_field [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html#type-cross-fields](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html#type-cross-fields) query. It works very well and is exactly what I need. However, in some rare circumstances the order of the results doesn't match my expectations. It turns out that the scoring of the first results is much higher than the score of the rest of the documents. I had a closer look at the explain statements and was surprised by the way the scores were calculated: for the first doc: { "value": 8.252264, "description": "fieldWeight in 998806, product of:", "details": [ { "value": 1, "description": "tf(freq=1.0), with freq of:", "details": [ { "value": 1, "description": "termFreq=1.0" } ] }, { "value": 8.252264, "description": "idf(docFreq=13182, maxDocs=18605118)" }, { "value": 1, "description": "fieldNorm(doc=998806)" } ] } and for the doc that is supposed to be first: { "value": 3.8485851, "description": "score(doc=700068,freq=1.0 = termFreq=1.0\n), product of:", "details": [ { "value": 0.46578622, "description": "queryWeight, product of:", "details": [ { "value": 18, "description": "boost" }, { "value": 8.262557, "description": "idf(docFreq=13047, maxDocs=18605118)" }, { "value": 0.0031318406, "description": "queryNorm" } ] }, { "value": 8.262557, "description": "fieldWeight in 700068, product of:", "details": [ { "value": 1, "description": "tf(freq=1.0), with freq of:", "details": [ { "value": 1, "description": "termFreq=1.0" } ] }, { "value": 8.262557, "description": "idf(docFreq=13047, maxDocs=18605118)" }, { "value": 1, "description": "fieldNorm(doc=700068)" } ] } ] } You can see that the queryWeight factor is missing in the calculation of the first doc, which leads to a much higher total score. I am no expert in it, but this seems to be a bug in my eyes. Or did I misunderstand something? You can find the query and the result list here: > Thanks for your help! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3526746f-d8a9-4344-978a-9240bbd38a13%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3526746f-d8a9-4344-978a-9240bbd38a13%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Christoph\_Lingg](https://avatars.discourse-cdn.com/v4/letter/c/5f9b8f/32.png) [@Christoph\_Lingg](https://discuss.elastic.co/u/Christoph_Lingg)\ **Post date:** [June 27, 2014, 1:17pm UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/2 "2014-06-27T13:17:01Z")
Another effect I do not understand ist the queryNorm which differs between documents, reading the documents I assumed them to be constant. From the lucene documentation [http://lucene.apache.org/core/3\_0\_3/api/all/org/apache/lucene/search/Similarity.html](http://lucene.apache.org/core/3_0_3/api/all/org/apache/lucene/search/Similarity.html) : > queryNorm(q) is a normalizing factor used to make scores between queries > comparable. This factor does not affect document ranking (since all ranked > documents are multiplied by the same factor), but rather just attempts to > make scores from different queries (or even different indexes) comparable. this is from the scoring of the first results { "value": 0.0059806756, "description": "queryNorm" } others have: { "value": 0.0031318406, "description": "queryNorm" } as the queryNorm finally _does_ affect document ranking i am asking myself if i am doing something wrong ... -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d258c287-a758-4165-8d0f-46a8bb950b8f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d258c287-a758-4165-8d0f-46a8bb950b8f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Christoph\_Lingg](https://avatars.discourse-cdn.com/v4/letter/c/5f9b8f/32.png) [@Christoph\_Lingg](https://discuss.elastic.co/u/Christoph_Lingg)\ **Post date:** [June 27, 2014, 4:22pm UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/3 "2014-06-27T16:22:20Z")
hm, after some investigations it turns out that queryNorm is related to the shard. I observed that only one of the five shard has a different query norm, all the others have equal ones. I will retry with only one shard to see if things are getting better. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9e96308e-ba99-499c-9f8e-64f2e7b088b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9e96308e-ba99-499c-9f8e-64f2e7b088b6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Stephane\_Bastian](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Stephane\_Bastian](https://discuss.elastic.co/u/Stephane_Bastian)\ **Post date:** [June 30, 2014, 7:42am UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/4 "2014-06-30T07:42:51Z")
Christoph, I'm wondering if the problem comes from the 'query type' parameter? > **[Understanding "Query Then Fetch" vs "DFS Query Then Fetch" | Elastic](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch)** > > In our last article on starts-with phrase matching, we ran into a situation where the returned scores were suspicious. As a refresher, here is the query in question:$ curl -XGET localhost:9200/startsw... Have you tried dfs\_query\_then\_fetch? Does it make any difference? All the best, Stéphane -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7b7745a4-5b6d-4b49-8c5b-d37e25619026%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7b7745a4-5b6d-4b49-8c5b-d37e25619026%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Christoph\_Lingg](https://avatars.discourse-cdn.com/v4/letter/c/5f9b8f/32.png) [@Christoph\_Lingg](https://discuss.elastic.co/u/Christoph_Lingg)\ **Post date:** [June 30, 2014, 8:53am UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/5 "2014-06-30T08:53:04Z")
Hi Stephane, yes I did try it but the results did not change. However, I reduced the number of shards from 5 to 1 and now the queryNorm is the same for every document. I learned that every shard is an independent lucene index and therefore different weights are likely to occur. However, the first strange behavior (missing queryWeight factor) still occurs from time to time, gracefully not too often. Cheers, Christoph -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3bb3bdc9-9709-410d-8609-7c276657ed67%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3bb3bdc9-9709-410d-8609-7c276657ed67%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:18am UTC](https://discuss.elastic.co/t/surprising-scoring-when-using-multi-matchs-cross-field/18371/6 "2017-07-06T01:18:56Z")