# Auto suggest with elasticsearch

**URL:** <https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763>\
**Category:** Elasticsearch\
**Created:** [July 4, 2011, 7:15am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763 "2011-07-04T07:15:51Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alexander\_Reelsen](https://avatars.discourse-cdn.com/v4/letter/a/85e7bf/32.png) [@Alexander\_Reelsen](https://discuss.elastic.co/u/Alexander_Reelsen)\
**Post date:** [July 4, 2011, 7:15am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/1 "2011-07-04T07:15:51Z")

</div>

Hi there,

I am currently trying to find the most elegant way to implement auto  
suggest feature for my elasticsearch instance.

Currently there is only an index with products, which includes product  
name, description, some image urls, some stock data etc...  
Now I want to implement auto suggest only for the product name.

The first question is: Is it useful to use this product index or to  
use a specific auto completion index for this? To keep the returned  
data as small as possible I do not want to return all the product data  
with each letter typed.

I am imagining an autocompletion solution like this and would love to  
get some input whether this is useful at all and could be implemented  
with elasticsearch:

1. Create a handler (not sure if its an index) on /suggest/
2. On any request like /suggest/foo or /suggest/foobar search for an  
entry in the suggest index(?) and return something like  
{  
key: "foo",  
suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
something: "else"  
}

So, would something like this be useful and can it be automatically  
created out of the product feed (or any feed to be more generic) to  
ensure it includes all my product names (would not need to be  
realtime, but updating once a day would be ok as well). Always adding  
all possible typing combinations manually sounds like quite some  
overhead, if there could be analyzers doing this work...

Thanks for any input!

--Alexander

---

<div class="post-metadata">

**Author:** ![Stephane\_Bastian](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Stephane\_Bastian](https://discuss.elastic.co/u/Stephane_Bastian)\
**Post date:** [July 4, 2011, 8:01am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/2 "2011-07-04T08:01:12Z")

</div>

Hello,

On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:

> Hi there,
> 
> I am currently trying to find the most elegant way to implement auto  
> suggest feature for my elasticsearch instance.
> 
> Currently there is only an index with products, which includes product  
> name, description, some image urls, some stock data etc...  
> Now I want to implement auto suggest only for the product name.
> 
> The first question is: Is it useful to use this product index or to  
> use a specific auto completion index for this? To keep the returned  
> data as small as possible I do not want to return all the product data  
> with each letter typed.
> 
> I am imagining an autocompletion solution like this and would love to  
> get some input whether this is useful at all and could be implemented  
> with elasticsearch:
> 
> 1. Create a handler (not sure if its an index) on /suggest/
> 2. On any request like /suggest/foo or /suggest/foobar search for an  
> entry in the suggest index(?) and return something like  
> {  
> key: "foo",  
> suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> something: "else"  
> }

ES is definitely a good candidate to provide auto-suggest type  
fonctionality.  
IMHO the first thing you've got to focus-on is to decide the type of  
auto-suggest you are looking for.  
In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
to be prefix-based. But is this always the case?  
Also, what do you expect if someone enters a word which does not match  
the prefix ? (for instance the user enters "francisco", do you expect  
"san francisco" to be displayed?)  
Another thing to consider is the expected behavior with sentences.  
On top of my head you also should decide whether or not we'll need to  
sort or highlight the results.

Based on this, you can make the right decision as to the most  
appropriate combination of Tokenizer/Filters/Queries to use.

If you want to implement a quick autocompletion I suggest that you look  
at the "text\_phrase\_prefix" query. it should give you a good head-start

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Hope this helps.

Since auto-suggest seems to be popular, how about if we enhance ES doc  
with the most common auto-suggest use cases and solutions? (with pros  
and cons, snippets of code, tokenizer/filter/query to use and such).  
If other people are interested, I can certainly contribute to it

> So, would something like this be useful and can it be automatically  
> created out of the product feed (or any feed to be more generic) to  
> ensure it includes all my product names (would not need to be  
> realtime, but updating once a day would be ok as well). Always adding  
> all possible typing combinations manually sounds like quite some  
> overhead, if there could be analyzers doing this work...
> 
> Thanks for any input!
> 
> --Alexander

---

<div class="post-metadata">

**Author:** ![Weiwei\_Wang](https://avatars.discourse-cdn.com/v4/letter/w/b19c9b/32.png) [@Weiwei\_Wang](https://discuss.elastic.co/u/Weiwei_Wang)\
**Post date:** [July 4, 2011, 8:37am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/3 "2011-07-04T08:37:57Z")

</div>

auto suggest can be implemented by add a field tokenized by  
StandardTokenizer(or other) and filter by EdgeNGramFilter or  
NGramFilter according to your project requirement.

You can look into the lucene contrib Spellchecker for some insight  
understanding.

and this article [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
should be helpful for understanding search suggestion.

On Jul 4, 4:01 pm, stephane [stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) wrote:

> Hello,
> 
> On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> 
> > Hi there,
> 
> > I am currently trying to find the most elegant way to implement auto  
> > suggest feature for my elasticsearch instance.
> 
> > Currently there is only an index with products, which includes product  
> > name, description, some image urls, some stock data etc...  
> > Now I want to implement auto suggest only for the product name.
> 
> > The first question is: Is it useful to use this product index or to  
> > use a specific auto completion index for this? To keep the returned  
> > data as small as possible I do not want to return all the product data  
> > with each letter typed.
> 
> > I am imagining an autocompletion solution like this and would love to  
> > get some input whether this is useful at all and could be implemented  
> > with elasticsearch:
> 
> > 1. Create a handler (not sure if its an index) on /suggest/
> > 2. On any request like /suggest/foo or /suggest/foobar search for an  
> > entry in the suggest index(?) and return something like  
> > {  
> > key: "foo",  
> > suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> > something: "else"  
> > }
> 
> ES is definitely a good candidate to provide auto-suggest type  
> fonctionality.  
> IMHO the first thing you've got to focus-on is to decide the type of  
> auto-suggest you are looking for.  
> In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> to be prefix-based. But is this always the case?  
> Also, what do you expect if someone enters a word which does not match  
> the prefix ? (for instance the user enters "francisco", do you expect  
> "san francisco" to be displayed?)  
> Another thing to consider is the expected behavior with sentences.  
> On top of my head you also should decide whether or not we'll need to  
> sort or highlight the results.
> 
> Based on this, you can make the right decision as to the most  
> appropriate combination of Tokenizer/Filters/Queries to use.
> 
> If you want to implement a quick autocompletion I suggest that you look  
> at the "text\_phrase\_prefix" query. it should give you a good head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> 
> Hope this helps.
> 
> Since auto-suggest seems to be popular, how about if we enhance ES doc  
> with the most common auto-suggest use cases and solutions? (with pros  
> and cons, snippets of code, tokenizer/filter/query to use and such).  
> If other people are interested, I can certainly contribute to it
> 
> > So, would something like this be useful and can it be automatically  
> > created out of the product feed (or any feed to be more generic) to  
> > ensure it includes all my product names (would not need to be  
> > realtime, but updating once a day would be ok as well). Always adding  
> > all possible typing combinations manually sounds like quite some  
> > overhead, if there could be analyzers doing this work...
> 
> > Thanks for any input!
> 
> > --Alexander

---

<div class="post-metadata">

**Author:** ![Michael\_McCandless](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Michael\_McCandless](https://discuss.elastic.co/u/Michael_McCandless)\
**Post date:** [July 4, 2011, 12:49pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/4 "2011-07-04T12:49:13Z")

</div>

Also, as of Lucene 3.3, the spellchecker modules  
(lucene/contrib/spellchecker) now has 3 implementations for  
auto-suggest, so in theory ES can just expose these?

Mike

[http://blog.mikemccandless.com](http://blog.mikemccandless.com)

On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang [ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) wrote:

> auto suggest can be implemented by add a field tokenized by  
> StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> NGramFilter according to your project requirement.
> 
> You can look into the lucene contrib Spellchecker for some insight  
> understanding.
> 
> and this article [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> should be helpful for understanding search suggestion.
> 
> On Jul 4, 4:01 pm, stephane [stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) wrote:
> 
> > Hello,
> > 
> > On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> > 
> > > Hi there,
> > 
> > > I am currently trying to find the most elegant way to implement auto  
> > > suggest feature for my elasticsearch instance.
> > 
> > > Currently there is only an index with products, which includes product  
> > > name, description, some image urls, some stock data etc...  
> > > Now I want to implement auto suggest only for the product name.
> > 
> > > The first question is: Is it useful to use this product index or to  
> > > use a specific auto completion index for this? To keep the returned  
> > > data as small as possible I do not want to return all the product data  
> > > with each letter typed.
> > 
> > > I am imagining an autocompletion solution like this and would love to  
> > > get some input whether this is useful at all and could be implemented  
> > > with elasticsearch:
> > 
> > > 1. Create a handler (not sure if its an index) on /suggest/
> > > 2. On any request like /suggest/foo or /suggest/foobar search for an  
> > > entry in the suggest index(?) and return something like  
> > > {  
> > > key: "foo",  
> > > suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> > > something: "else"  
> > > }
> > 
> > ES is definitely a good candidate to provide auto-suggest type  
> > fonctionality.  
> > IMHO the first thing you've got to focus-on is to decide the type of  
> > auto-suggest you are looking for.  
> > In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> > to be prefix-based. But is this always the case?  
> > Also, what do you expect if someone enters a word which does not match  
> > the prefix ? (for instance the user enters "francisco", do you expect  
> > "san francisco" to be displayed?)  
> > Another thing to consider is the expected behavior with sentences.  
> > On top of my head you also should decide whether or not we'll need to  
> > sort or highlight the results.
> > 
> > Based on this, you can make the right decision as to the most  
> > appropriate combination of Tokenizer/Filters/Queries to use.
> > 
> > If you want to implement a quick autocompletion I suggest that you look  
> > at the "text\_phrase\_prefix" query. it should give you a good head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> > 
> > Hope this helps.
> > 
> > Since auto-suggest seems to be popular, how about if we enhance ES doc  
> > with the most common auto-suggest use cases and solutions? (with pros  
> > and cons, snippets of code, tokenizer/filter/query to use and such).  
> > If other people are interested, I can certainly contribute to it
> > 
> > > So, would something like this be useful and can it be automatically  
> > > created out of the product feed (or any feed to be more generic) to  
> > > ensure it includes all my product names (would not need to be  
> > > realtime, but updating once a day would be ok as well). Always adding  
> > > all possible typing combinations manually sounds like quite some  
> > > overhead, if there could be analyzers doing this work...
> > 
> > > Thanks for any input!
> > 
> > > --Alexander

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [July 4, 2011, 7:33pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/5 "2011-07-04T19:33:35Z")

</div>

Hi,

Yea, doing auto suggest can be done in several manners. The first is simply executed a query against the relevant field, and only getting it back. That field can be analyzed (for example, using ngrams) if it make sense for the auto suggestion.

@mike: I saw the suggest module added, the problem with that is the fact that it requires a full rebuild and can't be dynamically updated to reflect changes in the index. This requires a system where periodic rebuilds are done, and personally, not a big fan of that :). It can become expensive to rebuild (file system cache invalidation), non intuitive (where it lacks real or near real timeness). Though, the FST based one is cool :). If an in memory auto suggest is required on a field, I was thinking of a non blocking trie based data structure (derived from ConcurrentSkipList), but that requires work 🙂

-shay.banon

On Monday, July 4, 2011 at 3:49 PM, Michael McCandless wrote:

> Also, as of Lucene 3.3, the spellchecker modules  
> (lucene/contrib/spellchecker) now has 3 implementations for  
> auto-suggest, so in theory ES can just expose these?
> 
> Mike
> 
> [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> 
> On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang \<[ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) ([mailto:ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com))\> wrote:
> 
> > auto suggest can be implemented by add a field tokenized by  
> > StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> > NGramFilter according to your project requirement.
> > 
> > You can look into the lucene contrib Spellchecker for some insight  
> > understanding.
> > 
> > and this article [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> > should be helpful for understanding search suggestion.
> > 
> > On Jul 4, 4:01 pm, stephane \<[stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > 
> > > Hello,
> > > 
> > > On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> > > 
> > > > Hi there,
> > > 
> > > > I am currently trying to find the most elegant way to implement auto  
> > > > suggest feature for my elasticsearch instance.
> > > 
> > > > Currently there is only an index with products, which includes product  
> > > > name, description, some image urls, some stock data etc...  
> > > > Now I want to implement auto suggest only for the product name.
> > > 
> > > > The first question is: Is it useful to use this product index or to  
> > > > use a specific auto completion index for this? To keep the returned  
> > > > data as small as possible I do not want to return all the product data  
> > > > with each letter typed.
> > > 
> > > > I am imagining an autocompletion solution like this and would love to  
> > > > get some input whether this is useful at all and could be implemented  
> > > > with elasticsearch:
> > > 
> > > > 1. Create a handler (not sure if its an index) on /suggest/
> > > > 2. On any request like /suggest/foo or /suggest/foobar search for an  
> > > > entry in the suggest index(?) and return something like  
> > > > {  
> > > > key: "foo",  
> > > > suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> > > > something: "else"  
> > > > }
> > > 
> > > ES is definitely a good candidate to provide auto-suggest type  
> > > fonctionality.  
> > > IMHO the first thing you've got to focus-on is to decide the type of  
> > > auto-suggest you are looking for.  
> > > In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> > > to be prefix-based. But is this always the case?  
> > > Also, what do you expect if someone enters a word which does not match  
> > > the prefix ? (for instance the user enters "francisco", do you expect  
> > > "san francisco" to be displayed?)  
> > > Another thing to consider is the expected behavior with sentences.  
> > > On top of my head you also should decide whether or not we'll need to  
> > > sort or highlight the results.
> > > 
> > > Based on this, you can make the right decision as to the most  
> > > appropriate combination of Tokenizer/Filters/Queries to use.
> > > 
> > > If you want to implement a quick autocompletion I suggest that you look  
> > > at the "text\_phrase\_prefix" query. it should give you a good head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> > > 
> > > Hope this helps.
> > > 
> > > Since auto-suggest seems to be popular, how about if we enhance ES doc  
> > > with the most common auto-suggest use cases and solutions? (with pros  
> > > and cons, snippets of code, tokenizer/filter/query to use and such).  
> > > If other people are interested, I can certainly contribute to it
> > > 
> > > > So, would something like this be useful and can it be automatically  
> > > > created out of the product feed (or any feed to be more generic) to  
> > > > ensure it includes all my product names (would not need to be  
> > > > realtime, but updating once a day would be ok as well). Always adding  
> > > > all possible typing combinations manually sounds like quite some  
> > > > overhead, if there could be analyzers doing this work...
> > > 
> > > > Thanks for any input!
> > > 
> > > > --Alexander

---

<div class="post-metadata">

**Author:** ![Michael\_McCandless](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Michael\_McCandless](https://discuss.elastic.co/u/Michael_McCandless)\
**Post date:** [July 4, 2011, 10:37pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/6 "2011-07-04T22:37:02Z")

</div>

Yeah I agree having to fully rebuild the suggest data structures  
periodically isn't great.

Mike

[http://blog.mikemccandless.com](http://blog.mikemccandless.com)

On Mon, Jul 4, 2011 at 3:33 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:

> Hi,  
> Yea, doing auto suggest can be done in several manners. The first is  
> simply executed a query against the relevant field, and only getting it  
> back. That field can be analyzed (for example, using ngrams) if it make  
> sense for the auto suggestion.  
> @mike: I saw the suggest module added, the problem with that is the fact  
> that it requires a full rebuild and can't be dynamically updated to reflect  
> changes in the index. This requires a system where periodic rebuilds are  
> done, and personally, not a big fan of that :). It can become expensive to  
> rebuild (file system cache invalidation), non intuitive (where it lacks real  
> or near real timeness). Though, the FST based one is cool :). If an in  
> memory auto suggest is required on a field, I was thinking of a non blocking  
> trie based data structure (derived from ConcurrentSkipList), but that  
> requires work 🙂  
> -shay.banon
> 
> On Monday, July 4, 2011 at 3:49 PM, Michael McCandless wrote:
> 
> Also, as of Lucene 3.3, the spellchecker modules  
> (lucene/contrib/spellchecker) now has 3 implementations for  
> auto-suggest, so in theory ES can just expose these?
> 
> Mike
> 
> [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> 
> On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang [ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) wrote:
> 
> auto suggest can be implemented by add a field tokenized by  
> StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> NGramFilter according to your project requirement.
> 
> You can look into the lucene contrib Spellchecker for some insight  
> understanding.
> 
> and this article  
> [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> should be helpful for understanding search suggestion.
> 
> On Jul 4, 4:01 pm, stephane [stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) wrote:
> 
> Hello,
> 
> On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> 
> Hi there,
> 
> I am currently trying to find the most elegant way to implement auto  
> suggest feature for my elasticsearch instance.
> 
> Currently there is only an index with products, which includes product  
> name, description, some image urls, some stock data etc...  
> Now I want to implement auto suggest only for the product name.
> 
> The first question is: Is it useful to use this product index or to  
> use a specific auto completion index for this? To keep the returned  
> data as small as possible I do not want to return all the product data  
> with each letter typed.
> 
> I am imagining an autocompletion solution like this and would love to  
> get some input whether this is useful at all and could be implemented  
> with elasticsearch:
> 
> 1. Create a handler (not sure if its an index) on /suggest/
> 2. On any request like /suggest/foo or /suggest/foobar search for an  
> entry in the suggest index(?) and return something like  
> {  
> key: "foo",  
> suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> something: "else"  
> }
> 
> ES is definitely a good candidate to provide auto-suggest type  
> fonctionality.  
> IMHO the first thing you've got to focus-on is to decide the type of  
> auto-suggest you are looking for.  
> In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> to be prefix-based. But is this always the case?  
> Also, what do you expect if someone enters a word which does not match  
> the prefix ? (for instance the user enters "francisco", do you expect  
> "san francisco" to be displayed?)  
> Another thing to consider is the expected behavior with sentences.  
> On top of my head you also should decide whether or not we'll need to  
> sort or highlight the results.
> 
> Based on this, you can make the right decision as to the most  
> appropriate combination of Tokenizer/Filters/Queries to use.
> 
> If you want to implement a quick autocompletion I suggest that you look  
> at the "text\_phrase\_prefix" query. it should give you a good  
> head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> 
> Hope this helps.
> 
> Since auto-suggest seems to be popular, how about if we enhance ES doc  
> with the most common auto-suggest use cases and solutions? (with pros  
> and cons, snippets of code, tokenizer/filter/query to use and such).  
> If other people are interested, I can certainly contribute to it
> 
> So, would something like this be useful and can it be automatically  
> created out of the product feed (or any feed to be more generic) to  
> ensure it includes all my product names (would not need to be  
> realtime, but updating once a day would be ok as well). Always adding  
> all possible typing combinations manually sounds like quite some  
> overhead, if there could be analyzers doing this work...
> 
> Thanks for any input!
> 
> --Alexander

---

<div class="post-metadata">

**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\
**Post date:** [July 5, 2011, 12:29pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/7 "2011-07-05T12:29:55Z")

</div>

But how much of a big deal is this in practice?

I suspect that most serious autosuggest users are populating their  
suggest, not directly from the terms, but based on query logs  
(probably with a lot of processing too!).  
It may be the case query logs are also in a lucene index, in which  
case the ability for suggest to pull from an index's terms is  
convenient, but I'm really suspicious if you can get a nice suggest  
based on the individual terms of the index itself.

I do happen to think you can get by ok this way for spellchecking,  
because its an easier problem: adding context gets you the "final  
inch" or last 5% or so, but doing it word-by-word is still ok.

all this being said, it might be possible to implement in a manner  
similar to lucene 4.0's directspellchecker, but performance will be  
tricky.  
in [[LUCENE-2507] automaton spellchecker - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-2507), I benchmarked  
the initial patch and it was unacceptable... only after Mike rewrote  
the terms dictionary one or two times did we start to see acceptable  
performance.

for suggest, the user might literally be waiting on this thing for  
every keystroke: and that scares me. maybe if we added a 'direct  
suggester' the perf would be ok for lots of people, but maybe it would  
require specialized data structures (e.g. terms index/dict impls) to  
get ok performance... but I'm personally a little discouraged from  
looking at this myself, because if ultimately you cannot get good  
_quality_ from the index itself, it could be a big waste of time.

On Mon, Jul 4, 2011 at 6:37 PM, Michael McCandless  
[mail@mikemccandless.com](mailto:mail@mikemccandless.com) wrote:

> Yeah I agree having to fully rebuild the suggest data structures  
> periodically isn't great.
> 
> Mike
> 
> [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> 
> On Mon, Jul 4, 2011 at 3:33 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> 
> > Hi,  
> > Yea, doing auto suggest can be done in several manners. The first is  
> > simply executed a query against the relevant field, and only getting it  
> > back. That field can be analyzed (for example, using ngrams) if it make  
> > sense for the auto suggestion.  
> > @mike: I saw the suggest module added, the problem with that is the fact  
> > that it requires a full rebuild and can't be dynamically updated to reflect  
> > changes in the index. This requires a system where periodic rebuilds are  
> > done, and personally, not a big fan of that :). It can become expensive to  
> > rebuild (file system cache invalidation), non intuitive (where it lacks real  
> > or near real timeness). Though, the FST based one is cool :). If an in  
> > memory auto suggest is required on a field, I was thinking of a non blocking  
> > trie based data structure (derived from ConcurrentSkipList), but that  
> > requires work 🙂  
> > -shay.banon
> > 
> > On Monday, July 4, 2011 at 3:49 PM, Michael McCandless wrote:
> > 
> > Also, as of Lucene 3.3, the spellchecker modules  
> > (lucene/contrib/spellchecker) now has 3 implementations for  
> > auto-suggest, so in theory ES can just expose these?
> > 
> > Mike
> > 
> > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > 
> > On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang [ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) wrote:
> > 
> > auto suggest can be implemented by add a field tokenized by  
> > StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> > NGramFilter according to your project requirement.
> > 
> > You can look into the lucene contrib Spellchecker for some insight  
> > understanding.
> > 
> > and this article  
> > [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> > should be helpful for understanding search suggestion.
> > 
> > On Jul 4, 4:01 pm, stephane [stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) wrote:
> > 
> > Hello,
> > 
> > On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> > 
> > Hi there,
> > 
> > I am currently trying to find the most elegant way to implement auto  
> > suggest feature for my elasticsearch instance.
> > 
> > Currently there is only an index with products, which includes product  
> > name, description, some image urls, some stock data etc...  
> > Now I want to implement auto suggest only for the product name.
> > 
> > The first question is: Is it useful to use this product index or to  
> > use a specific auto completion index for this? To keep the returned  
> > data as small as possible I do not want to return all the product data  
> > with each letter typed.
> > 
> > I am imagining an autocompletion solution like this and would love to  
> > get some input whether this is useful at all and could be implemented  
> > with elasticsearch:
> > 
> > 1. Create a handler (not sure if its an index) on /suggest/
> > 2. On any request like /suggest/foo or /suggest/foobar search for an  
> > entry in the suggest index(?) and return something like  
> > {  
> > key: "foo",  
> > suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> > something: "else"  
> > }
> > 
> > ES is definitely a good candidate to provide auto-suggest type  
> > fonctionality.  
> > IMHO the first thing you've got to focus-on is to decide the type of  
> > auto-suggest you are looking for.  
> > In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> > to be prefix-based. But is this always the case?  
> > Also, what do you expect if someone enters a word which does not match  
> > the prefix ? (for instance the user enters "francisco", do you expect  
> > "san francisco" to be displayed?)  
> > Another thing to consider is the expected behavior with sentences.  
> > On top of my head you also should decide whether or not we'll need to  
> > sort or highlight the results.
> > 
> > Based on this, you can make the right decision as to the most  
> > appropriate combination of Tokenizer/Filters/Queries to use.
> > 
> > If you want to implement a quick autocompletion I suggest that you look  
> > at the "text\_phrase\_prefix" query. it should give you a good  
> > head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> > 
> > Hope this helps.
> > 
> > Since auto-suggest seems to be popular, how about if we enhance ES doc  
> > with the most common auto-suggest use cases and solutions? (with pros  
> > and cons, snippets of code, tokenizer/filter/query to use and such).  
> > If other people are interested, I can certainly contribute to it
> > 
> > So, would something like this be useful and can it be automatically  
> > created out of the product feed (or any feed to be more generic) to  
> > ensure it includes all my product names (would not need to be  
> > realtime, but updating once a day would be ok as well). Always adding  
> > all possible typing combinations manually sounds like quite some  
> > overhead, if there could be analyzers doing this work...
> > 
> > Thanks for any input!
> > 
> > --Alexander

---

<div class="post-metadata">

**Author:** ![bjfish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bjfish/32/32676_2.png) [@bjfish](https://discuss.elastic.co/u/bjfish)\
**Post date:** [July 5, 2011, 2:58pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/8 "2011-07-05T14:58:19Z")

</div>

Here is a StumbleUpon blog article about search suggestions with ES  
that you might find useful if you have not seen it yet:  
[http://www.stumbleupon.com/su/6rCBMC/www.stumbleupon.com/devblog/searching-for-serendipity/](http://www.stumbleupon.com/su/6rCBMC/www.stumbleupon.com/devblog/searching-for-serendipity/)

On Jul 4, 2:15 am, Alexander Reelsen  
[alexander.reel...@googlemail.com](mailto:alexander.reel...@googlemail.com) wrote:

> Hi there,
> 
> I am currently trying to find the most elegant way to implement auto  
> suggest feature for my elasticsearch instance.
> 
> Currently there is only an index with products, which includes product  
> name, description, some image urls, some stock data etc...  
> Now I want to implement auto suggest only for the product name.
> 
> The first question is: Is it useful to use this product index or to  
> use a specific auto completion index for this? To keep the returned  
> data as small as possible I do not want to return all the product data  
> with each letter typed.
> 
> I am imagining an autocompletion solution like this and would love to  
> get some input whether this is useful at all and could be implemented  
> with elasticsearch:
> 
> 1. Create a handler (not sure if its an index) on /suggest/
> 2. On any request like /suggest/foo or /suggest/foobar search for an  
> entry in the suggest index(?) and return something like  
> {  
> key: "foo",  
> suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> something: "else"
> 
> }
> 
> So, would something like this be useful and can it be automatically  
> created out of the product feed (or any feed to be more generic) to  
> ensure it includes all my product names (would not need to be  
> realtime, but updating once a day would be ok as well). Always adding  
> all possible typing combinations manually sounds like quite some  
> overhead, if there could be analyzers doing this work...
> 
> Thanks for any input!
> 
> --Alexander

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [July 6, 2011, 1:06am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/9 "2011-07-06T01:06:32Z")

</div>

Heya Robert,

Its hard to answer on "suggestion" question, since suggestion can be interpreted in so many different manners... :).

So, I agree, for cases where the data is pretty much static or semi static, then the suggestion module is perfect, but there are many cases where its not... Suggestion module for user entered queries, that suggest popular queries falls nicely into one of those.

But, there are many other suggestion types. Starting from suggestion based on fields (like username, titles, or something similar), to more advance ones (geo based for example). Those lend more into the realtime scenario. Note, you can still build the suggestion from the fields (either fields that are not analyzed, or directly from "source", or, best, is "as you index"), so it does not necessarily works on analyzed terms.

-shay.banon

On Tuesday, July 5, 2011 at 3:29 PM, Robert Muir wrote:

> But how much of a big deal is this in practice?
> 
> I suspect that most serious autosuggest users are populating their  
> suggest, not directly from the terms, but based on query logs  
> (probably with a lot of processing too!).  
> It may be the case query logs are also in a lucene index, in which  
> case the ability for suggest to pull from an index's terms is  
> convenient, but I'm really suspicious if you can get a nice suggest  
> based on the individual terms of the index itself.
> 
> I do happen to think you can get by ok this way for spellchecking,  
> because its an easier problem: adding context gets you the "final  
> inch" or last 5% or so, but doing it word-by-word is still ok.
> 
> all this being said, it might be possible to implement in a manner  
> similar to lucene 4.0's directspellchecker, but performance will be  
> tricky.  
> in [[LUCENE-2507] automaton spellchecker - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-2507), I benchmarked  
> the initial patch and it was unacceptable... only after Mike rewrote  
> the terms dictionary one or two times did we start to see acceptable  
> performance.
> 
> for suggest, the user might literally be waiting on this thing for  
> every keystroke: and that scares me. maybe if we added a 'direct  
> suggester' the perf would be ok for lots of people, but maybe it would  
> require specialized data structures (e.g. terms index/dict impls) to  
> get ok performance... but I'm personally a little discouraged from  
> looking at this myself, because if ultimately you cannot get good  
> _quality_ from the index itself, it could be a big waste of time.
> 
> On Mon, Jul 4, 2011 at 6:37 PM, Michael McCandless  
> \<[mail@mikemccandless.com](mailto:mail@mikemccandless.com) ([mailto:mail@mikemccandless.com](mailto:mail@mikemccandless.com))\> wrote:
> 
> > Yeah I agree having to fully rebuild the suggest data structures  
> > periodically isn't great.
> > 
> > Mike
> > 
> > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > 
> > On Mon, Jul 4, 2011 at 3:33 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote:
> > 
> > > Hi,  
> > > Yea, doing auto suggest can be done in several manners. The first is  
> > > simply executed a query against the relevant field, and only getting it  
> > > back. That field can be analyzed (for example, using ngrams) if it make  
> > > sense for the auto suggestion.  
> > > @mike: I saw the suggest module added, the problem with that is the fact  
> > > that it requires a full rebuild and can't be dynamically updated to reflect  
> > > changes in the index. This requires a system where periodic rebuilds are  
> > > done, and personally, not a big fan of that :). It can become expensive to  
> > > rebuild (file system cache invalidation), non intuitive (where it lacks real  
> > > or near real timeness). Though, the FST based one is cool :). If an in  
> > > memory auto suggest is required on a field, I was thinking of a non blocking  
> > > trie based data structure (derived from ConcurrentSkipList), but that  
> > > requires work 🙂  
> > > -shay.banon
> > > 
> > > On Monday, July 4, 2011 at 3:49 PM, Michael McCandless wrote:
> > > 
> > > Also, as of Lucene 3.3, the spellchecker modules  
> > > (lucene/contrib/spellchecker) now has 3 implementations for  
> > > auto-suggest, so in theory ES can just expose these?
> > > 
> > > Mike
> > > 
> > > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > > 
> > > On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang \<[ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) ([mailto:ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com))\> wrote:
> > > 
> > > auto suggest can be implemented by add a field tokenized by  
> > > StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> > > NGramFilter according to your project requirement.
> > > 
> > > You can look into the lucene contrib Spellchecker for some insight  
> > > understanding.
> > > 
> > > and this article  
> > > [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> > > should be helpful for understanding search suggestion.
> > > 
> > > On Jul 4, 4:01 pm, stephane \<[stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote:
> > > 
> > > Hello,
> > > 
> > > On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> > > 
> > > Hi there,
> > > 
> > > I am currently trying to find the most elegant way to implement auto  
> > > suggest feature for my elasticsearch instance.
> > > 
> > > Currently there is only an index with products, which includes product  
> > > name, description, some image urls, some stock data etc...  
> > > Now I want to implement auto suggest only for the product name.
> > > 
> > > The first question is: Is it useful to use this product index or to  
> > > use a specific auto completion index for this? To keep the returned  
> > > data as small as possible I do not want to return all the product data  
> > > with each letter typed.
> > > 
> > > I am imagining an autocompletion solution like this and would love to  
> > > get some input whether this is useful at all and could be implemented  
> > > with elasticsearch:
> > > 
> > > 1. Create a handler (not sure if its an index) on /suggest/
> > > 2. On any request like /suggest/foo or /suggest/foobar search for an  
> > > entry in the suggest index(?) and return something like  
> > > {  
> > > key: "foo",  
> > > suggestions: ["foob", "fooba", "foobar", "foobored", "foo baz"],  
> > > something: "else"  
> > > }
> > > 
> > > ES is definitely a good candidate to provide auto-suggest type  
> > > fonctionality.  
> > > IMHO the first thing you've got to focus-on is to decide the type of  
> > > auto-suggest you are looking for.  
> > > In your example ("foob", "fooba", "foobar"...), the auto-suggest seems  
> > > to be prefix-based. But is this always the case?  
> > > Also, what do you expect if someone enters a word which does not match  
> > > the prefix ? (for instance the user enters "francisco", do you expect  
> > > "san francisco" to be displayed?)  
> > > Another thing to consider is the expected behavior with sentences.  
> > > On top of my head you also should decide whether or not we'll need to  
> > > sort or highlight the results.
> > > 
> > > Based on this, you can make the right decision as to the most  
> > > appropriate combination of Tokenizer/Filters/Queries to use.
> > > 
> > > If you want to implement a quick autocompletion I suggest that you look  
> > > at the "text\_phrase\_prefix" query. it should give you a good  
> > > head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> > > 
> > > Hope this helps.
> > > 
> > > Since auto-suggest seems to be popular, how about if we enhance ES doc  
> > > with the most common auto-suggest use cases and solutions? (with pros  
> > > and cons, snippets of code, tokenizer/filter/query to use and such).  
> > > If other people are interested, I can certainly contribute to it
> > > 
> > > So, would something like this be useful and can it be automatically  
> > > created out of the product feed (or any feed to be more generic) to  
> > > ensure it includes all my product names (would not need to be  
> > > realtime, but updating once a day would be ok as well). Always adding  
> > > all possible typing combinations manually sounds like quite some  
> > > overhead, if there could be analyzers doing this work...
> > > 
> > > Thanks for any input!
> > > 
> > > --Alexander

---

<div class="post-metadata">

**Author:** ![Stephane\_Bastian](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@Stephane\_Bastian](https://discuss.elastic.co/u/Stephane_Bastian)\
**Post date:** [July 6, 2011, 7:01am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/10 "2011-07-06T07:01:16Z")

</div>

Hi All,

As Shay pointed-out, auto-suggest type functionality must be handled  
differently based on the expected behavior.

Currently Elasticsearch can handle most 'auto-suggest' use cases and I  
believe most questions users have could be answered by some sort of FAQ  
or a section in the doc.  
Should we open a feature request to provide better doc around  
auto-suggest and start documenting basic use cases for new ES users?

Stephane Bastian

On Wed, 2011-07-06 at 04:06 +0300, Shay Banon wrote:

> Heya Robert,
> 
> ```
> Its hard to answer on "suggestion" question, since suggestion can
> 
> ```
> 
> be interpreted in so many different manners... :).
> 
> ```
> So, I agree, for cases where the data is pretty much static or
> 
> ```
> 
> semi static, then the suggestion module is perfect, but there are many  
> cases where its not... Suggestion module for user entered queries,  
> that suggest popular queries falls nicely into one of those.
> 
> ```
> But, there are many other suggestion types. Starting from
> 
> ```
> 
> suggestion based on fields (like username, titles, or something  
> similar), to more advance ones (geo based for example). Those lend  
> more into the realtime scenario. Note, you can still build the  
> suggestion from the fields (either fields that are not analyzed, or  
> directly from "source", or, best, is "as you index"), so it does not  
> necessarily works on analyzed terms.
> 
> -shay.banon  
> On Tuesday, July 5, 2011 at 3:29 PM, Robert Muir wrote:
> 
> > But how much of a big deal is this in practice?
> > 
> > I suspect that most serious autosuggest users are populating their  
> > suggest, not directly from the terms, but based on query logs  
> > (probably with a lot of processing too!).  
> > It may be the case query logs are also in a lucene index, in which  
> > case the ability for suggest to pull from an index's terms is  
> > convenient, but I'm really suspicious if you can get a nice suggest  
> > based on the individual terms of the index itself.
> > 
> > I do happen to think you can get by ok this way for spellchecking,  
> > because its an easier problem: adding context gets you the "final  
> > inch" or last 5% or so, but doing it word-by-word is still ok.
> > 
> > all this being said, it might be possible to implement in a manner  
> > similar to lucene 4.0's directspellchecker, but performance will be  
> > tricky.  
> > in [[LUCENE-2507] automaton spellchecker - ASF JIRA](https://issues.apache.org/jira/browse/LUCENE-2507), I benchmarked  
> > the initial patch and it was unacceptable... only after Mike rewrote  
> > the terms dictionary one or two times did we start to see acceptable  
> > performance.
> > 
> > for suggest, the user might literally be waiting on this thing for  
> > every keystroke: and that scares me. maybe if we added a 'direct  
> > suggester' the perf would be ok for lots of people, but maybe it  
> > would  
> > require specialized data structures (e.g. terms index/dict impls) to  
> > get ok performance... but I'm personally a little discouraged from  
> > looking at this myself, because if ultimately you cannot get good  
> > _quality_ from the index itself, it could be a big waste of time.
> > 
> > On Mon, Jul 4, 2011 at 6:37 PM, Michael McCandless  
> > [mail@mikemccandless.com](mailto:mail@mikemccandless.com) wrote:
> > 
> > > Yeah I agree having to fully rebuild the suggest data structures  
> > > periodically isn't great.
> > > 
> > > Mike
> > > 
> > > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > > 
> > > On Mon, Jul 4, 2011 at 3:33 PM, Shay Banon  
> > > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:
> > > 
> > > > Hi,  
> > > > Yea, doing auto suggest can be done in several manners. The  
> > > > first is  
> > > > simply executed a query against the relevant field, and only  
> > > > getting it  
> > > > back. That field can be analyzed (for example, using ngrams) if  
> > > > it make  
> > > > sense for the auto suggestion.  
> > > > @mike: I saw the suggest module added, the problem with that  
> > > > is the fact  
> > > > that it requires a full rebuild and can't be dynamically updated  
> > > > to reflect  
> > > > changes in the index. This requires a system where periodic  
> > > > rebuilds are  
> > > > done, and personally, not a big fan of that :). It can become  
> > > > expensive to  
> > > > rebuild (file system cache invalidation), non intuitive (where  
> > > > it lacks real  
> > > > or near real timeness). Though, the FST based one is cool :). If  
> > > > an in  
> > > > memory auto suggest is required on a field, I was thinking of a  
> > > > non blocking  
> > > > trie based data structure (derived from ConcurrentSkipList), but  
> > > > that  
> > > > requires work 🙂  
> > > > -shay.banon
> > > > 
> > > > On Monday, July 4, 2011 at 3:49 PM, Michael McCandless wrote:
> > > > 
> > > > Also, as of Lucene 3.3, the spellchecker modules  
> > > > (lucene/contrib/spellchecker) now has 3 implementations for  
> > > > auto-suggest, so in theory ES can just expose these?
> > > > 
> > > > Mike
> > > > 
> > > > [http://blog.mikemccandless.com](http://blog.mikemccandless.com)
> > > > 
> > > > On Mon, Jul 4, 2011 at 4:37 AM, Weiwei Wang  
> > > > [ww.wang.cs@gmail.com](mailto:ww.wang.cs@gmail.com) wrote:
> > > > 
> > > > auto suggest can be implemented by add a field tokenized by  
> > > > StandardTokenizer(or other) and filter by EdgeNGramFilter or  
> > > > NGramFilter according to your project requirement.
> > > > 
> > > > You can look into the lucene contrib Spellchecker for some  
> > > > insight  
> > > > understanding.
> > > > 
> > > > and this article  
> > > > [http://today.java.net/pub/a/today/2005/08/09/didyoumean.html](http://today.java.net/pub/a/today/2005/08/09/didyoumean.html)  
> > > > should be helpful for understanding search suggestion.
> > > > 
> > > > On Jul 4, 4:01 pm, stephane [stephane.bastian....@gmail.com](mailto:stephane.bastian....@gmail.com)  
> > > > wrote:
> > > > 
> > > > Hello,
> > > > 
> > > > On Mon, 2011-07-04 at 00:15 -0700, Alexander Reelsen wrote:
> > > > 
> > > > Hi there,
> > > > 
> > > > I am currently trying to find the most elegant way to implement  
> > > > auto  
> > > > suggest feature for my elasticsearch instance.
> > > > 
> > > > Currently there is only an index with products, which includes  
> > > > product  
> > > > name, description, some image urls, some stock data etc...  
> > > > Now I want to implement auto suggest only for the product name.
> > > > 
> > > > The first question is: Is it useful to use this product index or  
> > > > to  
> > > > use a specific auto completion index for this? To keep the  
> > > > returned  
> > > > data as small as possible I do not want to return all the  
> > > > product data  
> > > > with each letter typed.
> > > > 
> > > > I am imagining an autocompletion solution like this and would  
> > > > love to  
> > > > get some input whether this is useful at all and could be  
> > > > implemented  
> > > > with elasticsearch:
> > > > 
> > > > 1. Create a handler (not sure if its an index) on /suggest/
> > > > 2. On any request like /suggest/foo or /suggest/foobar search  
> > > > for an  
> > > > entry in the suggest index(?) and return something like  
> > > > {  
> > > > key: "foo",  
> > > > suggestions: [ "foob", "fooba", "foobar", "foobored", "foo  
> > > > baz" ],  
> > > > something: "else"  
> > > > }
> > > > 
> > > > ES is definitely a good candidate to provide auto-suggest type  
> > > > fonctionality.  
> > > > IMHO the first thing you've got to focus-on is to decide the  
> > > > type of  
> > > > auto-suggest you are looking for.  
> > > > In your example ("foob", "fooba", "foobar"...), the auto-suggest  
> > > > seems  
> > > > to be prefix-based. But is this always the case?  
> > > > Also, what do you expect if someone enters a word which does not  
> > > > match  
> > > > the prefix ? (for instance the user enters "francisco", do you  
> > > > expect  
> > > > "san francisco" to be displayed?)  
> > > > Another thing to consider is the expected behavior with  
> > > > sentences.  
> > > > On top of my head you also should decide whether or not we'll  
> > > > need to  
> > > > sort or highlight the results.
> > > > 
> > > > Based on this, you can make the right decision as to the most  
> > > > appropriate combination of Tokenizer/Filters/Queries to use.
> > > > 
> > > > If you want to implement a quick autocompletion I suggest that  
> > > > you look  
> > > > at the "text\_phrase\_prefix" query. it should give you a good  
> > > > head-starthttp://www.elasticsearch.org/guide/reference/query-dsl/text-query.html
> > > > 
> > > > Hope this helps.
> > > > 
> > > > Since auto-suggest seems to be popular, how about if we enhance  
> > > > ES doc  
> > > > with the most common auto-suggest use cases and solutions? (with  
> > > > pros  
> > > > and cons, snippets of code, tokenizer/filter/query to use and  
> > > > such).  
> > > > If other people are interested, I can certainly contribute to it
> > > > 
> > > > So, would something like this be useful and can it be  
> > > > automatically  
> > > > created out of the product feed (or any feed to be more generic)  
> > > > to  
> > > > ensure it includes all my product names (would not need to be  
> > > > realtime, but updating once a day would be ok as well). Always  
> > > > adding  
> > > > all possible typing combinations manually sounds like quite some  
> > > > overhead, if there could be analyzers doing this work...
> > > > 
> > > > Thanks for any input!
> > > > 
> > > > --Alexander

---

<div class="post-metadata">

**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\
**Post date:** [July 6, 2011, 7:57am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/11 "2011-07-06T07:57:16Z")

</div>

On Tue, Jul 5, 2011 at 9:06 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:

> Heya Robert,  
> Its hard to answer on "suggestion" question, since suggestion can be  
> interpreted in so many different manners... :).  
> So, I agree, for cases where the data is pretty much static or semi  
> static, then the suggestion module is perfect, but there are many cases  
> where its not... Suggestion module for user entered queries, that suggest  
> popular queries falls nicely into one of those.  
> But, there are many other suggestion types. Starting from suggestion  
> based on fields (like username, titles, or something similar), to more  
> advance ones (geo based for example). Those lend more into the realtime  
> scenario. Note, you can still build the suggestion from the fields (either  
> fields that are not analyzed, or directly from "source", or, best, is "as  
> you index"), so it does not necessarily works on analyzed terms.  
> -shay.banon

Well, for these simplistic suggestion types on unanalyzed content,  
there is no problem, you just need to select a Lookup implementation  
that supports add() (e.g. TST, Jaspell, but not FST)

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [July 6, 2011, 3:49pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/12 "2011-07-06T15:49:20Z")

</div>

> Well, for these simplistic suggestion types on unanalyzed content,  
> there is no problem, you just need to select a Lookup implementation  
> that supports add() (e.g. TST, Jaspell, but not FST)  
> Not sure they are that simplistic :), since it needs to be concurrent (even a read write lock around it will be expensive) and allow for "deletion" (as in reduce counter).

---

<div class="post-metadata">

**Author:** ![rmuir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rmuir/32/44949_2.png) [@rmuir](https://discuss.elastic.co/u/rmuir)\
**Post date:** [July 6, 2011, 7:08pm UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/13 "2011-07-06T19:08:21Z")

</div>

On Wed, Jul 6, 2011 at 11:49 AM, Shay Banon  
[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote:

> Well, for these simplistic suggestion types on unanalyzed content,  
> there is no problem, you just need to select a Lookup implementation  
> that supports add() (e.g. TST, Jaspell, but not FST)
> 
> Not sure they are that simplistic :), since it needs to be concurrent (even  
> a read write lock around it will be expensive) and allow for "deletion" (as  
> in reduce counter).

I think thats really overkill? You could always have terms in lucene  
that have all deleted docs, which will affect spellcheck, too.

but in both cases, for a normal search engine suggest, the following hold true:

- normally you filter only high-freq terms (HighFrequencyDictionary or  
thresholdFrequency in lucene), so the chance of spellcorrecting a  
only-deleted docs term is minimized.
- similar for suggest, the chance is minimal, e.g. if you are building  
from say terms of the past N-days of query logs I don't think you gain  
much by going to so much effort to expunge these all-deleted-terms for  
things people stopped querying on, e.g. a periodic rebuild is  
sufficient for your suggest to represent the past N-days trend.

But in general, this is the use case that the suggest/spellcheck  
framework is geared towards (along with supplying floating point  
quality weights, etc). If instead you want to do a suggest that acts  
more like a primitive prefix query on a transactional database and  
less like a search engine, I think using edge ngrams is the way to go,  
as lucene will take care of that stuff for you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:01am UTC](https://discuss.elastic.co/t/auto-suggest-with-elasticsearch/4763/14 "2017-07-06T04:01:32Z")

</div>