Replicate same quality of search via console as if I'm using Enterprise search

[es_com (Nik)]

I'm trying to get same results when I use search through the console as if I'm using Enterprise search through UI.

Enterprise works so much better.

Here are some of the approaches I took:

POST /my-index/_search
{
  "size": 17,
  "query": {
    "nested": {
      "path": "passages",
      "query": {
        "sparse_vector": {
          "field": "passages.vector", 
          "inference_id": ".elser_model_2_linux-x86_64", 
          "query": "some random question here" 
        }
      },
      "inner_hits": {
        "_source": ["passages.text"], 
        "highlight": {
          "fields": {
            "passages.text": {} 
          }
        },
        "sort": [
          "_score"  //
        ]
      }
    }
  }
}

GET my-index/_search
{
  "query": {
    "nested": {
      "path": "passages",
      "query": {
        "bool": {
          "must": [
            {
              "text_expansion": {
                "passages.sparse.tokens": {
                  "model_id": ".elser_model_2_linux-x86_64",
                  "model_text": "some question here"
                }
              }
            },
            {
              "match": {
                "passages.text": "some question here"
              }
            }
          ]
        }
      },
      "inner_hits": {
        "_source": {
          "excludes": ["passages.sparse"]
        }
      }
    }
  }
}

Does anyone know what magic is hidden in the Enterprise search so that it works that good?

[Kathleen_DeRusso]

Hey there, when you say Enterprise Search I assume you mean App Search?

If that's the case, I'd suggest checking out the App Search Explain API - this shows the exact Elasticsearch query that's being run for Enterprise Search. You could then modify it for your semantic query. That and the custom analyzers/mappings that App Search creates would be worth looking into.

You may also be interested in this Python notebook. The notebook is geared towards users migrating from Enterprise Search to native Elasticsearch but you may find it helpful to see the mappings and examples of how to incorporate semantic search.

Hope that helps!

[es_com (Nik)]

Thank you for your response @Kathleen_DeRusso . Just to be safe we're on the same page here, were you referring to this search?

[Kathleen_DeRusso]

Ah, no. That's different than Enterprise Search

Depending on how you created your index, you may have some text mappings automatically created for you defining things like custom analyzers and subfields. But I think the document explorer is a pretty lightweight query and won't be using ELSER. It's probably just a match query TBH.

ELSER and semantic search in general really shines when you're trying to ask a question or get the contextual meaning of a query. So for example, if you're looking for the best places to see the Northern Lights and want to get results for Iceland and Alaska. If you're looking for straight up keyword match results only though you might not be happy with the expanded recall that ELSER provides.

If you want to use ELSER but boost the match results, you could boost the match clause in the Boolean query and play with the values that get you the best results too.

[es_com (Nik)]

That's the problem. You can see from my initial comment that I have been using both sparse vector search and text expansion.... but the document explorer outperformed both and always gave more accurate results, with any type of queries I tried (simple, complex, etc).

Additionally, it still looks to me that document explorer is using Enterprise search

[Kathleen_DeRusso]

Ah, that path is legacy naming in the UI, Enterprise Search refers to the products App Search and Workplace Search. Sorry for the confusion!

I understand you're not happy with the ELSER results, but other than using match queries I really can't give any more specific advice without more information.