### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 15, 2024, 11:37pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/1 "2024-03-15T23:37:30Z")
Hi, I use Enterprise elasticsearch and ingest documents using web crawler. The data is pre-processed using an Inference pipeline which creates vectors for title, meta\_description fields before ingesting the docs to the indexes. I also assign a field named "priority" for all the documents based on the page URL (crawled by the web crawler).
Things were fine for English markets when using ELSER embeddings + script\_score to customize the documents score, but since now we have started working with non-English locales and creating embeddings using E5 model, I'm struggling to find an example for running approximate KNN search as well as customizing the score values using a script\_score. Also note that I cannot use RRF as well, since I need highlighted feilds.
Below is the score calculation done for English markets with ELSER embeddings:
```auto
"query": {
"script_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": {{query_string_temp}},
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
{
"text_expansion": {
"ml.inference.title_expanded.predicted_value": {
"model_text": {{query_string_temp}},
"model_id": ".elser_model_2_linux-x86_64",
"boost": 5.0
}
}
},
{
"text_expansion": {
"ml.inference.meta_description_expanded.predicted_value": {
"model_text": {{query_string_temp}},
"model_id": ".elser_model_2_linux-x86_64",
"boost": 4.0
}
}
},
{
"text_expansion": {
"ml.inference.meta_keywords_expanded.predicted_value": {
"model_text": {{query_string_temp}},
"model_id": ".elser_model_2_linux-x86_64",
"boost": 4.0
}
}
},
{
"text_expansion": {
"ml.inference.headings_expanded.predicted_value": {
"model_text": {{query_string_temp}},
"model_id": ".elser_model_2_linux-x86_64",
"boost": 4.0
}
}
},
{
"bool": {
"boost": 1.0
}
},
{
"bool": {
"boost": 1.0
}
}
],
"minimum_should_match": "1",
"boost": 1.0
}
},
"script": {
"source": "if(doc['priority'].size() > 0){ return _score*(1-(doc['priority'].value-1)/10) }",
"lang": "painless"
}
}
},
"min_score": 50.0
```
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 18, 2024, 8:41pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/2 "2024-03-18T20:41:09Z")
Hey @ravi_kumar_mvs
You should be able to use: [Knn query | Elasticsearch Guide [8.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-knn-query.html) since its just a query, it can be part of a script\_score query.
Now, this doesn't have the query\_builder logic in it yet like the regular top-level `knn` object. But this was recently merged, so should be in 8.14:
>
>
> Make it possible to perform KNN queries by supplying \`model\_text\` and \`model\_id\`… instead of the \`query\_vector\`.
>
> This makes use of a \`QueryVectorBuilder\`. Supplying a \`text\_embedding\` \`query\_vector\_builder\` with \`model\_text\` and \`model\_id\` instead of the \`query\_vector\` will result in the generation of a \`query\_vector\` by calling inference (during query rewrite) on the specified \`model\_id\` with the supplied \`model\_text\`.
> This is consistent with the way query vectors are built from \`model\_id\` / \`model\_text\` in \`KnnSearchBuilder\` (DFS phase).
>
> Sample query:
> \`\`\`json
> {
> "query": {
> "knn" : {
> "field": "embedding",
> "num\_candidates": 10,
> "query\_vector\_builder": {
> "text\_embedding": {
> "model\_id": "bert\_base",
> "model\_text": "lucene is all you need"
> }
> }
> }
> }
> }
> \`\`\`
> See also https://docs.google.com/document/d/12SYyHbPbCzhPYQ65HiRMesANObvzWDrBDm0Qn9QSwFQ/edit#heading=h.r3mn4wd2it4e
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 18, 2024, 9:55pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/3 "2024-03-18T21:55:43Z")
Hi BenTrent,
Thank you for your time and reply.
Yes, I have tried that. And it works when I don't use script\_score or function\_score queries to modify the score.
For example, the below query works:
```auto
{
"query": {
"multi_match": {
"query": "{{query_string_temp}}",
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
"knn": [
{
"field": "ml.inference.vector_title.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100,
"boost":5
},
{
"field": "ml.inference.vector_headings.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100,
"boost":4
},
{
"field": "ml.inference.vector_meta_description.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100,
"boost":4
},
{
"field": "ml.inference.vector_meta_keywords.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100,
"boost":4
}
]
}
```
But this one doesn't work:
```auto
{
"from": 0,
"size": 10,
"query": {
"script_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "{{query_string_temp}}",
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
{
"knn": [
{
"field": "ml.inference.vector_title.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_headings.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_description.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_keywords.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"k": 5,
"num_candidates": 100
}
]
}
]
}
},
"script": {
"source": "if(doc['priority'].size() > 0){ return _score*(1-(doc['priority'].value-1)/10) }",
"lang": "painless"
}
}
},
"min_score": 50.0
}
```
Also my elasticsearch cloud cluster is on : 8.11.4
If I understand your comment, when top level "knn" queries are supported (with 8.14) I'll be able to use them for script\_score or function\_score, correct? If so, any other alternatives to support customizing score at the moment?
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 19, 2024, 12:36pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/4 "2024-03-19T12:36:24Z")
Ah, what I am talking about is this:
[Knn query | Elasticsearch Guide [8.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-knn-query.html)
Which is is available in 8.12+
But, you are using the `query_vector_builder` interface as well, which isn't available for the query until 8.14.
So, currently, there isn't a way to do what you want to do exactly until 8.14 is released.
What your query would look like in 8.14:
```auto
{
"from": 0,
"size": 10,
"query": {
"script_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "{{query_string_temp}}",
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
{
"knn": {
"field": "ml.inference.vector_title.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"num_candidates": 100
}
},
{
"knn": {
"field": "ml.inference.vector_headings.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"num_candidates": 100
}
},
{
"knn": {
"field": "ml.inference.vector_meta_description.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"num_candidates": 100
}
},
{
"knn": {
"field": "ml.inference.vector_meta_keywords.predicted_value",
"query_vector_builder": {
"text_embedding": {
"model_id": "multilingual-e5-small",
"model_text": "{{query_string_temp}}"
}
},
"num_candidates": 100
}
}
]
}
}
},
"script": {
"source": "if(doc['priority'].size() > 0){ return _score*(1-(doc['priority'].value-1)/10) }",
"lang": "painless"
}
},
"min_score": 50.0
}
```
You could do this in 8.12 if you replace the `"query_vector_builder":` entries with `query_vector` and the already embedded text.
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 19, 2024, 1:00pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/5 "2024-03-19T13:00:18Z")
Really appreciate your help.
So as I understand, the best option for now is to make 2 network calls. 1 call to fetch query embeddings and the second call to make the actual query call .
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 20, 2024, 1:33pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/6 "2024-03-20T13:33:26Z")
Hi BenTrent,
Unfortunately even query\_vector queries are not working with 8.11.4. I have tried the below which worked :
```auto
{
"query": {
"multi_match": {
"query": "{{query_string_temp}}",
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
"knn": [
{
"field": "ml.inference.vector_title.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_headings.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_description.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_keywords.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
}
]
}
```
But the below still fails:
```auto
"query": {
"script_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "{{query_string_temp}}",
"fields": [
"body_content^3.0",
"headings^4.0",
"meta_description^4.0",
"meta_keywords.text^4.0",
"title^5.0"
]
}
},
{
"knn": [
{
"field": "ml.inference.vector_title.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_headings.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_description.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
},
{
"field": "ml.inference.vector_meta_keywords.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
}
]
}
]
}
},
"script": {
"source": "if(doc['priority'].size() > 0){ return _score*(1-(doc['priority'].value-1)/10) }",
"lang": "painless"
}
}
},
"min_score": 50.0
```
{{query\_string\_temp\_vectors}} is the actual vector with the same dims as the query model expects. [0.04206765815615654,-0.03618289530277252,..etc]
Error message: "reason": "[knn] query malformed, no start\_object after query name",
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 20, 2024, 1:52pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/7 "2024-03-20T13:52:21Z")
> [@ravi\_kumar\_mvs](#):
>
> Unfortunately even query\_vector queries are not working with 8.11.4. I have tried the below which worked :
Correct, knn in the `query` clause is only available in 8.12+
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 21, 2024, 8:41am UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/8 "2024-03-21T08:41:47Z")
Hi BenTrent,
We have upgraded to 8.12.2 and lost all our web crawler indexes. We will evaluate why we lost them.
Also unfortunately 8.12.2 still doesn't support array of knn requests with query\_vector.
For example, below query works:
```auto
{
"size" : 3,
"query": {
"bool": {
"should": [
{
"match": {
"title": {
"query": "{{query_string_temp}}",
"boost": 1
}
}
},
{
"knn": {
"field": "ml.inference.vector_title.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"num_candidates": 10,
"boost": 1
}
}
]
}
},
"_source":["title"]
}
```
But not the below one:
```auto
{
"size": 3,
"query": {
"bool": {
"should": [
{
"match": {
"title": {
"query": "{{query_string_temp}}",
"boost": 1
}
}
},
{
"knn": [
{
"field": "ml.inference.vector_title.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"num_candidates": 10,
"boost": 1
},
{
"field": "ml.inference.vector_headings.predicted_value",
"query_vector": {{query_string_temp_vectors}},
"k": 5,
"num_candidates": 100
}
]
}
]
}
},
"_source": [
"title"
]
}
```
Error message:
```auto
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[knn] query malformed, no start_object after query name",
"line": 15,
"col": 28
}
],
"type": "x_content_parse_exception",
"reason": "[15:28] [bool] failed to parse field [should]",
"caused_by": {
"type": "parsing_exception",
"reason": "[knn] query malformed, no start_object after query name",
"line": 15,
"col": 28
}
},
"status": 400
```
Looks like I'm blocked. Any suggestions?
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 21, 2024, 11:10am UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/9 "2024-03-21T11:10:44Z")
Sorry about your difficulties.
Each `knn` query is an individual query. See my example (shortened for brevity):
```auto
{
"from": 0,
"size": 10,
"query": {
"script_score": {
"query": {
"bool": {
"should": [
{
"knn": {
"field": "ml.inference.vector_title.predicted_value",
"query_vector": [1,2,3],
"num_candidates": 100
}
},
{
"knn": {
"field": "ml.inference.vector_headings.predicted_value",
"query_vector": [1,2,3],
"num_candidates": 100
}
},
...
]
}
}
},
"script": {
"source": "if(doc['priority'].size() > 0){ return _score*(1-(doc['priority'].value-1)/10) }",
"lang": "painless"
}
},
"min_score": 50.0
}
```
You can see many examples here on this particular doc page: [Knn query | Elasticsearch Guide [8.12] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-knn-query.html)
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 21, 2024, 11:37am UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/10 "2024-03-21T11:37:34Z")
Thank you. We will try this with 2 network calls.
Any possibility of achieving this in 1 Elasticsearch API call using 8.12.2? My be painless scripts or some other way?
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 21, 2024, 12:14pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/11 "2024-03-21T12:14:41Z")
Not in 8.12. In 8.14 the `knn` query will be brought into parity with the top level knn object.
---
### Author:  [@ravi\_kumar\_mvs](https://discuss.elastic.co/u/ravi_kumar_mvs)
#### Post date: [March 21, 2024, 2:21pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/12 "2024-03-21T14:21:42Z")
Oh. Ok. May be we are too early into it. We will then use ingestion \_simulate endpoint to form the embeddings of the user query and then pass the embeddings to the actual search template. Looks like that's the only option for now to execute the whole flow within elasticsearch platform.
Also any estimated date for the 8.14 release on elastic cloud ?
---
### Author:  [@BenTrent](https://discuss.elastic.co/u/BenTrent)
#### Post date: [March 21, 2024, 2:33pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/13 "2024-03-21T14:33:44Z")
> [@ravi\_kumar\_mvs](#):
>
> Also any estimated date for the 8.14 release on elastic cloud ?
We don't have any estimates on its release date.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 18, 2024, 2:34pm UTC](https://discuss.elastic.co/t/elasticsearch-aproximate-knn-search-using-custom-score-calculation-using-a-custom-priority-field/355524/14 "2024-04-18T14:34:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.