How to create a search request with multiple knn fields using the Node Client

Elastic Stack Elasticsearch
,
1

My first post here - I'm using elasticsearch 8.15.0 Node client.

I have an index with multiple dense-vector fields, and I want to search on them all.

With the python client, I am able to do this.
{query: {bool: {should:[{knn:{knn spec...}}]}}

According to the api docs I should be able to do this with v8.15.0
{knn: [{knn spec...}]}

The node client insists that the knn property be specficied as a body parameter separate from the query like this.
search ({query, knn, ...})

Using the approach, where the knn specs are included in a query.bool{}, (which works with Python), returns an error with the Node client.

Using the API documented approach with a knn array as a separate body argument to the Node client returns this error
Unknown key for a START_ARRAY in [knn]

What is correct approach with the Node client for a query with multiple knn fields?

thank you

2

Hi @Bruce_Mcpherson !

knn query and knn section allow a single field to be used.

You could combine multiple knn queries using a bool query:

{
  "query": {
    "bool": {
      "should": [
        {
          "knn": {
            "field": "field1",
            ...
          }
        },
        {
          "knn": {
            "field": "field2",
            ...
          }
        }
      ]
    }
  }
}

That will allow to adjust the score via individually boosting each query.

You can also use RRF to combine the queries.

3

Thanks for the response, Carlos.

That was the approach I used with the Python client and it worked fine. However with the Node client, I get this error

illegal_argument_exception: [knn] queries cannot be provided directly, use the [knn] body parameter instead

This is what led me supplying the knn as a top level body argument rather than as part of a query as described here.
multiple knn fields

I haven't come across rrf before (I see it's in preview) - but I'll take a look.

In the meantime, is there any reason you can think of that knn as a body argument with an array would not work as described returning this error

Unknown key for a START_ARRAY in [knn]

It does work with a single knn.

thank you

4

Hi @Bruce_Mcpherson , welcome to our community.

1- Can you share the exact error message you're receiving when trying to run the query with multiple knn fields? This will help identify if the issue is due to query formatting or something else.

2-Have you tried using a bool.should query to combine multiple knn searches into a single request? This approach allows you to run multiple knn queries on different vector fields and combine their results.

5

Hi Alex

The message for bool/should was

illegal_argument_exception: [knn] queries cannot be provided directly, use the [knn] body parameter instead

And for knn array method:

Unknown key for a START_ARRAY in [knn]

However - it was all my bad. I am running 8.15.0 client, but an earlier v8 version es server on kubernetes. I've just upgraded my kube server to 8.15.0 to match the client. Both methods now work!

Thank you both for your excellent and prompt responses.

6

Added language-clients