# cosineSimilarity: script score query returned an invalid score for doc:

**URL:** <https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931>\
**Category:** Elasticsearch\
**Created:** [January 6, 2020, 5:58pm UTC](https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931 "2020-01-06T17:58:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lok63](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lok63/32/64812_2.png) [@lok63](https://discuss.elastic.co/u/lok63)\
**Post date:** [January 6, 2020, 5:58pm UTC](https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931/1 "2020-01-06T17:58:04Z")

</div>

I have a strange issue with the \_search API. I saw other people in this forum which had similar issues and the problem for them was that they had negative values. However in my case I am using dense vectors. Moreover my script\_score is trying to calculate the cosine and I thought that cosine was returning negative values, however I am adding 1.0 to the score to make sure that it will never be negative.

Just to make sure that my dense vector don't have any issues I tried to do the dot product manually and it works. But when I try to use the cosineSimilarity with ES I get the following error.

**Elasticsearch version** : 7.4.2  
**lucene\_version** : 8.2.0  
**java version** : 1.8.0

I have an index called " **my\_index**" where it contains a field called **embeddings**. Its datatype is the following:

```json
"embeddings": {
    "type": "dense_vector",
    "dims": 300
},

```

Every singe item contains a vector and its not null

GET my\_index/\_search

```json
{
  "query": {
    "script_score": {
      "query" : {
          "match_all": {}
        },
      "script": {
        "source": "cosineSimilarity(params.queryVector, doc['embeddings']) + 1.0",
        "params": {
          "queryVector": "[my_300_Dimenion_Vector]"
        }
      }
    }
  }
}

```

The error looks like this:

```json
{
    "error": {
        "root_cause": [
            {
                "type": "exception",
                "reason": "script score query returned an invalid score: NaN for doc: 9"
            },
            {
                "type": "exception",
                "reason": "script score query returned an invalid score: NaN for doc: 178"
            },
            {
                "type": "exception",
                "reason": "script score query returned an invalid score: NaN for doc: 24"
            },
            {
                "type": "exception",
                "reason": "script score query returned an invalid score: NaN for doc: 44"
            },
            {
                "type": "exception",
                "reason": "script score query returned an invalid score: NaN for doc: 60"
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "webcontent_emb",
                "node": "eIrwXaCYRrWjjOdUJkW66A",
                "reason": {
                    "type": "exception",
                    "reason": "script score query returned an invalid score: NaN for doc: 9"
                }
            },
            {
                "shard": 1,
                "index": "webcontent_emb",
                "node": "aTC9SRxaRIK4gPdqCpEQ-w",
                "reason": {
                    "type": "exception",
                    "reason": "script score query returned an invalid score: NaN for doc: 178"
                }
            },
            {
                "shard": 2,
                "index": "webcontent_emb",
                "node": "eIrwXaCYRrWjjOdUJkW66A",
                "reason": {
                    "type": "exception",
                    "reason": "script score query returned an invalid score: NaN for doc: 24"
                }
            },
            {
                "shard": 3,
                "index": "webcontent_emb",
                "node": "aTC9SRxaRIK4gPdqCpEQ-w",
                "reason": {
                    "type": "exception",
                    "reason": "script score query returned an invalid score: NaN for doc: 44"
                }
            },
            {
                "shard": 4,
                "index": "webcontent_emb",
                "node": "jsHhvQSzRO-zkM--fH1JdA",
                "reason": {
                    "type": "exception",
                    "reason": "script score query returned an invalid score: NaN for doc: 60"
                }
            }
        ]
    },
    "status": 500
}

```

---

<div class="post-metadata">

**Author:** ![lok63](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lok63/32/64812_2.png) [@lok63](https://discuss.elastic.co/u/lok63)\
**Post date:** [January 7, 2020, 2:44pm UTC](https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931/2 "2020-01-07T14:44:09Z")

</div>

The problem was based on the dense vector values.  
It seems like I had vectors filled with zeros. Thats why cosineSimilarity was returning NaN values. After those entries were removed, cosine worked again

---

<div class="post-metadata">

**Author:** ![Julie\_Tibshirani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/julie_tibshirani/32/55628_2.png) [@Julie\_Tibshirani](https://discuss.elastic.co/u/Julie_Tibshirani)\
**Post date:** [January 9, 2020, 6:09pm UTC](https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931/3 "2020-01-09T18:09:04Z")

</div>

Hello @lok63, thanks for raising this point. At first I thought it may be a bug, but the behavior actually seems correct to me -- cosine similarity is not defined if one of the vectors is all zeros. If you have any ideas for how we could better handle this case, feel free to open a GitHub issue to start a discussion.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 6, 2020, 6:09pm UTC](https://discuss.elastic.co/t/cosinesimilarity-script-score-query-returned-an-invalid-score-for-doc/213931/4 "2020-02-06T18:09:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.