Potential bug in painless execute API since 7.5

[Denne]

I'm getting a strange error when using the painless execute API with cosineSimilarity. This only happens with version 7.5+. With version 7.4 this is not happening for me.

Here a minimal working example that uses the examples from the official documentation:

POST /_scripts/painless/_execute  
{  
  "script": {  
    "source": "cosineSimilarity(params.query_vector, 'query_vector') + 1.0",  
    "params": {  
      "query_vector": [3, 3.5, -0.1]  
    }  
  },  
  "context": "score",  
  "context_setup": {  
    "index": "my-index",  
    "document": {  
      "doc_vector": [4, 3.4, -0.2]  
    }  
  }  
}

Version used for this example: 7.8

Links to the documentation:
https://www.elastic.co/guide/en/elasticsearch/painless/current/painless-execute-api.html
https://www.elastic.co/guide/en/elasticsearch/reference/7.8/query-dsl-script-score-query.html#_dense_vector_functions

The exception I'm getting:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "script_exception",
        "reason" : "runtime error",
        "script_stack" : [
          "org.elasticsearch.script.ScoreScript._getIndexVersion(ScoreScript.java:204)",
          "org.elasticsearch.xpack.vectors.query.ScoreScriptUtils$DenseVectorFunction.getEncodedVector(ScoreScriptUtils.java:103)",
          "org.elasticsearch.xpack.vectors.query.ScoreScriptUtils$CosineSimilarity.cosineSimilarity(ScoreScriptUtils.java:179)",
          "cosineSimilarity(params.query_vector, 'skill_set_embedding') + 1.0",
          "                                      ^---- HERE"
        ],
        "script" : "cosineSimilarity(params.query_vector, 'skill_set_embedding') + 1.0",
        "lang" : "painless",
        "position" : {
          "offset" : 38,
          "start" : 0,
          "end" : 66
        }
      }
    ],
    "type" : "script_exception",
    "reason" : "runtime error",
    "script_stack" : [
      "org.elasticsearch.script.ScoreScript._getIndexVersion(ScoreScript.java:204)",
      "org.elasticsearch.xpack.vectors.query.ScoreScriptUtils$DenseVectorFunction.getEncodedVector(ScoreScriptUtils.java:103)",
      "org.elasticsearch.xpack.vectors.query.ScoreScriptUtils$CosineSimilarity.cosineSimilarity(ScoreScriptUtils.java:179)",
      "cosineSimilarity(params.query_vector, 'skill_set_embedding') + 1.0",
      "                                      ^---- HERE"
    ],
    "script" : "cosineSimilarity(params.query_vector, 'skill_set_embedding') + 1.0",
    "lang" : "painless",
    "position" : {
      "offset" : 38,
      "start" : 0,
      "end" : 66
    },
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "index version can not be looked up!"
    }
  },
  "status" : 400
}

Using the painless script in a normal search query works just fine. It only seems to be a problem when using the cosineSimilarity function in the painless execute API. I suspect it has something to do with the following MR which introduced the exception: https://github.com/elastic/elasticsearch/pull/46294

Since the issue tracker on github mentions to first open a topic in the forum if the bug is not confirmed I only posted it here at the moment. If someone can replicate it I can open an issue on github for this.

[Jack_Conradson]

Thank you for this great breakdown. It does look to indeed be the case that these methods were accidentally removed from the execute API. I agree they should be available in some form and will discuss with my team the best way to get them back in there.