Is it possible getting parent doc's score in nested query?

Elastic Stack Elasticsearch
1

For example, I have an index with following mapping:

{
    "mappings": {
      "docs": {
        "properties": {
          "QueryClicks": {
            "type": "nested",
            "properties": {
              "Count": {
                "type": "long"
              },
              "Term": {
                "type": "string",
                "fields": {
                  "lowercaseraw": {
                    "type": "string",
                    "analyzer": "lowercase_keyword"
                  }
                }
              }
            }
          },
          "Title": {
            "type": "string"
          }
        }
      }
    }
}

And example document:

{
  "Title": "The Heart of the Elastic Stack"
  "QueryClicks": [
      { "Term": "elastic stack", "Count": 100},
      { "Term": "elastic", "Count": 50},
      { "Term": "hard of the elastic", "Count": 200},
  ]
}

Now I want to get all documents which match query terms in Title field, and boost the score by QueryClicks.Count if the query term exactly match the corresponding QueryClicks.Term field.

Following is my initial query DSL:

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {"Title": "elastic stack"}
        }
      ],
     "should": [
        {
          "nested": {
            "path": "QueryClicks",
            "query": {
              "function_score": {
                "query": {
                  "match": {"QueryClicks.Term.lowercaseraw": "elastic stack"}
                },
                "functions": [{
                  "script_score": {
                    "script": "log(doc['QueryClicks.Count'].value*4)"
                  }
                }],
                "boost_mode": "replace"
              }
            }
          }
        }
     ]
    }
  }
}

It somehow works, for certain queryterm, if it match one of the QueryClicks.Term, extra scores will be added to the whole socre of the document.

But not perfect. What I want is to multiply the nested function score (that is, log(doc['QueryClicks.Count'].value*4) ) with the parent document's score which calculated in must clause.

If I can get the parent doc's score, then I can do something like this:

"script": "log(doc['QueryClicks.Count'].value*4) * _parent_score"

Is it possible?

Or any other approach to solve my requirement?

2

Anyone has suggestion?

3

I would try wraping the portion under must into a function score_query and convert the should part into a function of that function score query. So, you would end up with two function_score queries, one inside another.

4

Thanks!
I tried to build query with function_score inside another function_score, but could not work out a workable query.

Could you give me an simple example of function_score inside another function_score?

5

Could you show me the query that you tried and that didn't work?

6 
{
  "query" : {
    "function_score" : {
      "query" : {
        "match" : {
          "Title" : "elastic stack"
        }
      },
      "functions" : [{
          "filter" : {
            "nested" : {
              "path" : "QueryClicks",
              "query" : {
                "function_score" : {
                  "query" : {
                    "match" : {
                      "QueryClicks.Term.lowercaseraw" : "elastic stack"
                    }
                  },
                  "functions" : [{
                      "script_score" : {
                        "script" : "10000"
                      }
                    }
                  ],
                  "boost_mode" : "replace"
                }
              }
            }
          },
          "weight" : 1
        }
      ],
      "boost_mode" : "multiply"
    }
  }
}

To verify if it works, in the inside function_score, I simply set the score to 10000, and expect the final score will be around 10000, but it is actually around 3.

7

Could you execute it with the explain flag to see why?

8

Here is the explain output of first matched doc:

"_score" : 3.2159193,
"_explanation" : {
  "value" : 3.2159193,
  "description" : "sum of:",
  "details" : [{
      "value" : 3.2159193,
      "description" : "weight(Title:elastic stack in 104357) [PerFieldSimilarity], result of:",
      "details" : [{
          "value" : 3.2159193,
          "description" : "fieldWeight in 104357, product of:",
          "details" : [{
              "value" : 1,
              "description" : "tf(freq=1.0), with freq of:",
              "details" : [{
                  "value" : 1,
                  "description" : "termFreq=1.0",
                  "details" : []
                }
              ]
            }, {
              "value" : 3.2159193,
              "description" : "idf(docFreq=15520, maxDocs=142325)",
              "details" : []
            }, {
              "value" : 1,
              "description" : "fieldNorm(doc=104357)",
              "details" : []
            }
          ]
        }
      ]
    }, {
      "value" : 0,
      "description" : "match on required clause, product of:",
      "details" : [{
          "value" : 0,
          "description" : "# clause",
          "details" : []
        }, {
          "value" : 0.31095308,
          "description" : "_type:docs, product of:",
          "details" : [{
              "value" : 1,
              "description" : "boost",
              "details" : []
            }, {
              "value" : 0.31095308,
              "description" : "queryNorm",
              "details" : []
            }
          ]
        }
      ]
    }
  ]
}

It missed following like output from my original must + should query:

{
  "value": 10000,
  "description": "Score based on child doc range from 138429 to 138546",
  "details": []
}

Seems the nested function score is not executed?

9

I think I know why.

Coz the nested function_score is wrapped in "filter", so no score...

But if I changed "filter" to "query", I got parse error.

I read the ES reference, seems no way to wrap a query in functions clause of a function_score query.

Any suggestion?

10

Yep, you are right, that wouldn't really work. Sorry about that. I played with it a bit more and I don't see any reasonable solution here. The only crazy solution that comes to mind at the moment is to go back to the bool query wrap the must part into a function_score and calculate log(_score) in the script, then calculate log(log(doc['QueryClicks.Count'].value*4)) in should. Considering that log(xz)=log(x) + log(y) you can replace multiplication with addition that takes place in bool and since log is a strictly increasing function for positive x you can, theoretically, use log of the score instead of score for comparison. Practically speaking, I am not sure if this solution would work giving relatively low resolution of float score.

11

Queries wrapped by the nested query can't access the score of the parent
document, because at the time of scoring the parent document hasn't been
matched yet and therefor its score hasn't been computed yet.

12

Thanks you all.
So any other workaround except Igor's amazing solution of using math?

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.