Painless script - variable logging for scoring analysis

[peterkimmy (Peter Kim)]

Hello,

I am using a painless script to implement a custom scoring function while querying the ES index, that's being used as a basis for our recommendation engine. While calculating the final score in the painless script, I use a product of intermediate variables such as recency and uniqueness, calculated within the painless script.

Now, it is trivial to get the final scores of the top documents as they are returned within the query response. However, for detailed analysis, I'm trying to find a way to also get the intermediate variables' values (recency and uniqueness as in the above example). I understand these painless variables only exist within the context of the painless script, which does not have standard REPL setup. So is there really no way to access these painless variables? Has anyone found a workaround to do this? Thanks!

E.g. If I have the following simplified painless script:

def recency = 1/doc['date'].value
def uniqueness = doc['ctr].value

return recency * uniqueness

In the final ES response, I get the final script scores i.e. recency * uniqueness . However, I also want to know what the intermediate variables are i.e. recency and uniqueness

Best,
Peter

[rjernst (Ryan Ernst)]

Typically in Lucene this would be done with an Explanation for the query/score. However, this isn't currently exposed in painless scoring scripts.

As a workaround, you could have a field script that returns a string explanation of your own, given that it looks like your entire score is based on your own values. The overhead would be minimal since it would only run on the top N docs.

[rjernst (Ryan Ernst)]

@peterkimmy

for detailed analysis

Can you clarify what you mean by detailed analysis here? Will this be something you want to find as a one off for a single document, or return for every document? If for a single document, I think we can allow score scripts to manipulate the explanation returned by the explain api.