We have 3 stage ranking phase in our search engine. We use BM25 with Distance Decay function and Freshness Decay function as the first phase. Then for the second stage, we use LTR plugin and do a rescoring with a window_size. Third stage we have another boosting mechanism implemented using function_score. Both LTR and and Boosting phase use the same window_size. Here is the implementation of the Boosting using function_score.
{
"window_size": {{rescore.window_size}},
"query": {
"rescore_query": {
"function_score": {
"boost_mode":"sum",
"score_mode": "sum",
"script_score" : {
"script" : {
{{#rescore.ym.params}}
"params": {
"min": {{rescore.ym.params.min}},
"w_x": {{rescore.ym.params.w_x}},
"w_y": {{rescore.ym.params.w_y}},
"default_y": {{rescore.ym.params.default_y}},
},
{{/rescore.ym.params}}
{{#rescore.ym.use_add_func}}
"source": "params.min + (doc['ValueA'].empty ? 0 : params.w_x * doc['ValueA'].value) + params.w_y * (doc['ValueB'].empty ? params.default_y : doc['ValueB'].value)"
{{/rescore.ym.use_add_func}}
{{^rescore.ym.use_add_func}}
"source": "0"
{{/rescore.ym.use_add_func}}
}
}
}
}
}
}
The math function in the function_score use two fields in the document and generate the source to be use during the ranking. I calculate the manually what the source should get during ranking time and looked at the results but it doesn't look like final ranking is preserving the boosting or have gotten enough boosting. Following image shows elastic document score and then manually calculated value. This show case that boosting didn't really take place (there a clusters) .
Questions:
Is it due to baseline ranking? or is due to LTR and baseline contributing and the boosting done in function_score is too small?
In addition to above I have tried multiply as boosting along with replace neither option generated rank order that reflect the score calculated in the function.