Hi!
I use for a very specific use case script fields for scoring different attributes of documents using custom painless scripts, where the query context is used for filtering non-relevant documents (in the example below replaced with a match_all
query for simplicity).
Here is an example:
GET my_index/_search
{
"query": {
"match_all": {}
},
"script_fields": {
"attribute_1": {
"script": {
"lang": "painless",
"source": "cosineSimilarity(params.query_vector, 'dense_vector') + 1.0",
"params": {
"query_vector": [ ... ]
}
}
},
"attribute_2": {
"script": {
"lang": "painless",
"source": "...",
"params": {
"some_param": ...
}
}
}
}
}
This example does not work because cosineSimilarity
isn't allowed in the script_fields
context. The following error is returned:
...
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Unknown call [cosineSimilarity] with [2] arguments."
}
This makes it impossible to use vector embeddings (e.g. sentence embeddings) to calculate the similarity regarding different attributes and return these scores in the results.
Now my question: Why isn't this possible? I couldn't find a legitimate reason. Also: is there a workaround to get multiple scores for each hit using vector function?