Issues with scoring and query boost

Elastic Stack Elasticsearch
1

Hi,

We are currently migrating from lucene to elasticsearch. We rely on lucene
scoring to correctly sort our search results. We tried to use the scoring
of elasticsearch for the same.
But we are facing few issues with the algo, esp the boost parameter.

We query on multiple fields with different boost for each field.
For example... for the field "nametoken" we gave a boost of 5. But when we
look at the query explanation, we are getting the following:

13.09477 = (MATCH) sum of:
13.09477 = (MATCH) weight(nameTokens:somename^15625.0 in 28989),
product of:
0.9999997 = queryWeight(nameTokens:somename^15625.0), product of:
15625.0 = boost
13.094774 = idf(docFreq=36, maxDocs=6620571)
4.8874444E-6 = queryNorm
13.094774 = (MATCH) fieldWeight(nameTokens:somename in 28989),
product of:
1.0 = tf(termFreq(nameTokens:somename)=1)
13.094774 = idf(docFreq=36, maxDocs=6620571)
1.0 = fieldNorm(field=nameTokens, doc=28989)
0.5 = coord(1/2)

How did the boost increase to that. This does not happen in lucene.
We are using "dfs_query_then_fetch" search type. Index has 5 shards and 1
replica. Using 0.20.2 version of elasticsearch

Thanks
Rohit

--

2

I noticed the following behavior :

The scoring seems to change with each query though there is no change to
the index. This is mainly due to issues with boost parameter
The first query gives the correct results. But if the same query is fired
mulitple time the results change

For the query : with sort as score sort
Explanations for 2 returned docs and given below. Want to understand why is
the score changing. Am I doing something wrong

"bool" : {
"must" : {
"bool" : {
"should" : [ {
"bool" : {
"must" : {
"query_string" : {
"query" : ""microsoft"",
"fields" : [ "nameTokens" ],
"analyzer" : "analyzerWithoutStop"
}
},
"boost" : 5.0
}
}, {
"bool" : {
"should" : {
"query_string" : {
"query" : ""insideview.com"",
"fields" : [ "companyDomain" ],
"analyzer" : "analyzerWithoutStop"
}
},
"boost" : 10.0
}
} ]
}
}
}
}

Explanation for first query :

3.1991732 = (MATCH) product of:
6.3983464 = (MATCH) sum of:
6.3983464 = (MATCH) weight(companyDomain:insideview.com^10.0 in 223),
product of:
0.93694496 = queryWeight(companyDomain:insideview.com^10.0), product
of:
10.0 = boost
6.8289456 = idf(docFreq=2, maxDocs=1020)
0.013720199 = queryNorm
6.8289456 = (MATCH) fieldWeight(companyDomain:insideview.com in 223),
product of:
1.0 = tf(termFreq(companyDomain:insideview.com)=1)
6.8289456 = idf(docFreq=2, maxDocs=1020)
1.0 = fieldNorm(field=companyDomain, doc=223)
0.5 = coord(1/2)

0.89017844 = (MATCH) product of:
1.7803569 = (MATCH) sum of:
1.7803569 = (MATCH) weight(nameTokens:microsoft^5.0 in 209), product of:
0.3494771 = queryWeight(nameTokens:microsoft^5.0), product of:
5.0 = boost
5.0943446 = idf(docFreq=16, maxDocs=1020)
0.013720199 = queryNorm
5.0943446 = (MATCH) fieldWeight(nameTokens:microsoft in 209), product
of:
1.0 = tf(termFreq(nameTokens:microsoft)=1)
5.0943446 = idf(docFreq=16, maxDocs=1020)
1.0 = fieldNorm(field=nameTokens, doc=209)
0.5 = coord(1/2)

For 2nd query:

2.5471714 = (MATCH) product of:
5.0943427 = (MATCH) sum of:
5.0943427 = (MATCH) weight(nameTokens:microsoft^*15625.0 *in 209),
product of:
0.99999964 = queryWeight(nameTokens:microsoft^15625.0), product of:
15625.0 = boost
5.0943446 = idf(docFreq=16, maxDocs=1020)
1.2562947E-5 = queryNorm
5.0943446 = (MATCH) fieldWeight(nameTokens:microsoft in 209), product
of:
1.0 = tf(termFreq(nameTokens:microsoft)=1)
5.0943446 = idf(docFreq=16, maxDocs=1020)
1.0 = fieldNorm(field=nameTokens, doc=209)
0.5 = coord(1/2)

0.0029293338 = (MATCH) product of:
0.0058586677 = (MATCH) sum of:
0.0058586677 = (MATCH) weight(companyDomain:insideview.com^10.0 in
223), product of:
8.5791684E-4 = queryWeight(companyDomain:insideview.com^10.0),
product of:
10.0 = boost
6.8289456 = idf(docFreq=2, maxDocs=1020)
1.2562947E-5 = queryNorm
6.8289456 = (MATCH) fieldWeight(companyDomain:insideview.com in 223),
product of:
1.0 = tf(termFreq(companyDomain:insideview.com)=1)
6.8289456 = idf(docFreq=2, maxDocs=1020)
1.0 = fieldNorm(field=companyDomain, doc=223)
0.5 = coord(1/2)

--