Query suggest not working on app search - elastic index backed engines

Elastic Search
1

Hi,
I am running 8.4.1 of App Search and see that query_suggest does not work for engines created by elastic backends (i.e. non app search engines)

No results return

Can you please advise / provide a workaround ?
Thanks!

2

Hey @metalshanked ,

Thanks for raising this! I've dug in a little bit, and this looks like a bug. The Elasticsearch Index Engines feature is still in beta, so we really appreciate you being an early adopter, and helping us to smooth the rough edges. I'll file an issue, and we'll get it fixed shortly.

As a workaround, you can create a standard engine, and issue a query suggest API call to it with the X-Enterprise-Search-Debug: true header set. This will cause Enterprise Search to print out the exact Elasticsearch queries it is using behind the scenes to power your request. You can take a look at those, and modify them to be something you can call directly against the Elasticsearch index you are using for your Elasticsearch Index Engine.

Thanks again for reporting this issue!

1 Like
3

Thanks @Sean_Story.
Liking app search so far but the major limitation has been the 100 docs per request limit for ingest.
That was the reason to try out elasticsearch based indices so we can bulk index into them.

I see the app search indices have special mappings like .prefix etc.
so as a workaround to have the best of both worlds (fast bulk indexing AND app search features) here is what i did.

  1. Let app search create the engine and the associated index .ent-search-engine-documents-xxxxx

  2. Load data using vanilla Elastic bulk load helpers to these indices,

  3. Now we can use raw elastic AND app search via the engine interface or the raw indices.

No real need to use the wrapper feature search_es_search since raw elastic calls are anyways superior to search_es_search

4

@metalshanked as long as you're populating all of your subfields when indexing directly to the app search documents index, that's probably OK. You'll miss out on App Search's analytics and record keeping of you indexing those documents, but the same would be true if you were using pure elasticsearch.

The 100 docs per request limit can be worked around by using concurrency, and having multiple threads send 100 docs at a time. But yes, this is a common criticism. If you have a support relationship with Elastic, don't hesitate to file an Enhancement Request to ask us to increase that limit, or make it configurable.

5

Sure thanks @Sean_Story . Yes, using multithreading et al is still an issue since the documents go through the Enterprise Search nodes instead of directly loading the Elastic data nodes
which is quite slow and also tends to freeze up enterprise search many a times.

Client [100 docs x 10 workers ] --> Index [Enterprise Search/App Search Nodes] --> Elastic Data Nodes

vs

Client [2000 docs x 10 workers ]  --> Elastic Data Nodes

On a topic of querying,
Below are some issue i see is while running RAW elastic queries agains the App Search indices,

  1. match_phrase RAW Elastic queries do not work against the App Search indices
field:[myfield] was indexed without position data; cannot run PhraseQuery
  1. There is no keyword type set to the indices by app search
    so term queries fail unless we do myfield.enum
6

multithreading et al is still an issue since the documents go through the Enterprise Search nodes instead of directly loading the Elastic data nodes
which is quite slow and also tends to freeze up enterprise search many a times

Fair. This is one of the main motivations for why we're starting to work towards allowing App Search to interface directly with Elasticsearch indices, so that the Enterprise Search server can be decoupled from your ingestion process. But we are early in developing that feature. Thanks for continuing to provide feedback, so that we can mature it!

  1. match_phrase RAW Elastic queries do not work against the App Search indices

This surprises me. I'm checking with the team.

  1. There is no keyword type set to the indices by app search
    so term queries fail unless we do myfield.enum

This is expected behavior. If you look at the mappings for your index, you will see that <field>.enum is a keyword type.

7

Sure np!
Update: adding a .stem to the field in the search query now seems to work with match phrase.
like myfield.stem:"national park"
As per the doc below. The main text field myfield can't be used for phrase searches.

So I will have to ensure in the API calls to add the .stem suffix for phrase searches

1 Like
8

@Sean_Story - One other challange i am currently facing is to add a lowercase normalizer to the .enum field.

Since i cannot update the index settings to add the below to the index created by app search

"Can't update non dynamic settings [[index.analysis.normalizer.keyword_lowercase.type, index.analysis.normalizer.keyword_lowercase.filter]] for open indices [[.ent-search-engine-documents

was wondering if there is any way to make the .enum or keyword searches on app search indices case insensitive when directly querying the indices

9

Update: Figured out a way to add the lowercase normalizer.

in the index mapping, we can update the default enum mapping on the app search index with
"normalizer": "lowercase"

 "enum": {
                "normalizer": "lowercase",
                "ignore_above": 2048,
                "type": "keyword"
10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.