Confusion regarding elasticsearch enterprise search and app search

[elitzur_e]

Hi. after playing around in elastic cloud. i would like to setup a real (enterprise grade) solution for a big site.

in the instance i made there is a category "enterprise search" in that there is (among other things) Elasticsearch and app search.

so in app search i made a new engine and chose to use the web crawler. all looks good. it has its ow settings that are useful for the site.

then i wanted to use ESRE - so i saw i need to add an index - so i assumed the engine i created (would have a index) and i would have pipeline options. but there isn't.

so when i chose this time in the menu - Elasticsearch and created a new index and chose the web crawler - which seems to be working the same way.

only that is an index and in app search it is an engine. so I'm confused - i want to be able to use things like searchui, synonyms etc. but also use AI Powered thing that need an index. help

[Sean_Story]

Hi @elitzur_e - thanks for posting!

I'm sorry you're running into some confusion, but it makes me happy to hear that you're wanting to use so many of our features!

As you're discovering, Elastic Enterprise Search offers several different "flavors" of very similar concepts. We have two crawler implementations - the App Search Crawler (which writes directly to an Engine) and the Elastic Web Crawler (which writes directly to an Elasticsearch index). While the App Search Crawler actually can write documents through an Elasticsearch Ingest Pipeline (see these docs) it's definitely a lot more straightforward to do that through the Elastic Web Crawler, which will help you create your pipeline and even leverage the ELSER text-expansion model through the UI.

i want to be able to use things like searchui, synonyms etc.

search-ui has an Elasticsearch connector, though it's in technical preview. See the docs for it here.

There's also the option to build an App Search Engine off of an Elasticsearch Index. This lets you leverage both the Elastic Web Crawler AND the feature set of App Search (alongside the GA search-ui integration). HOWEVER, App Search does not currently have support for ESRE. You can further work around this by using Elasticsearch Search on your App Search Engine. Combined with the Search Explain API, you can see how App Search would form an Elasticsearch request itself, then you can augment that as a starting point to add in your own AI-powered query components and submit them via Elasticsearch APIs.

Sound convoluted? It is. App Search was designed to obfuscate this kind of query tuning, and "advanced feature" capabilities. But as we've found more folks who both want the simplicity of a good OOTB search, with the power to customize at a low level, we've started to work on a new product, Search Applications.

These make it programmatically much easier to submit a simple query string, but have it powered by a much more advanced Elasticsearch Query.

There's not a Search UI connector for Search Applications (yet), but there is this Search Application Client that can be used to help you start building your UI.

As far a synonyms go, stay tuned for our 8.10 release. The pre-release docs for Synonym search in Elasticsearch can be found here: Search with synonyms | Elasticsearch Guide [8.10] | Elastic

Also coming in the 8.10 release, "query rules", can be found here: Searching with query rules | Elasticsearch Guide [8.10] | Elastic and they'll enable curations on steroids as an Elasticsearch-first capability.