Execute Cosine Similarity inside script_score function if field is present in the document

Elastic Stack Elasticsearch
1

I am trying to use cosine similarity in script_score function. The query is breaking when the dense vector field is missing in the document against which I am trying to measure similarity is missing.

I spent a lot of time searching how to check if the field is present in document or not, but couldn't succeed.

I tried:

  1. checking with doc['field_name'] == null
  2. checking with doc['field_name'].size() == 0
  3. checking with doc['field_name'].value == null

Query i am using is

POST /sidx-4111c0fc-a8ba-523c-9851-34a2b803643b/_search/
{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "must": {
            "multi_match": {
              "query": "nri customer bank loan",
              "fields": [],
              "fuzziness": "AUTO"
            }
          }
        }
      },
      "functions": [
        {
          "script_score": {
            "script": {
              "source": "double score =0; score = doc['dense_vector_field'] == null ?0: cosineSimilarity(params.queryVector, 'dense_vector_field'); if(score>=0.8 && score<=1.0){return 10000+score;} else if(score>=0.60 && score<0.80){return score+1000;} else{return score+100}",
              "params": {
                "qv": [1,1,0,1]
              }
            }
          }
        }
      ],
      "boost_mode": "sum"
    }
  }
}

i am getting following error

  "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "No field found for [dense_vector_field] in mapping with types []"
          }
        }

can someone please help me

2

Sorry for a late reply:

You can check if a document is missing a field myvector by doc['myvector'].size() == 0.
For example, for a query something like this:

GET myindex/_search
{
  "query": {
    "script_score" : {
      "query" : {
        "match_all" : {}
      },
      "script" : {
        "source" : "doc['myvector'].size() == 0 ? 1 : cosineSimilarity(params.qv, 'myvector')",
        "params": {
          "qv": [1,1,0,1] 
        }
      }
    }
  }
}
3

Achieved it using containsKey and size functions, posting it so that someone may find it helpful

sample query :
"""
double score = 0;
if((doc.containsKey(field_name)!=false) && (doc[field_name].size()>0))
{score= cosineSimilarity(params.query_vector, field_name);}
"""

4

thanks for the reply, size function helped in getting solution for my usecase along with containsKey function

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.