# nGram performance **URL:** \ **Category:** Elasticsearch\ **Created:** [July 25, 2013, 4:35pm UTC](https://discuss.elastic.co/t/ngram-performance/12943 "2013-07-25T16:35:45Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Michael\_Lockwood](https://avatars.discourse-cdn.com/v4/letter/m/ecc23a/32.png) [@Michael\_Lockwood](https://discuss.elastic.co/u/Michael_Lockwood)\ **Post date:** [July 25, 2013, 4:35pm UTC](https://discuss.elastic.co/t/ngram-performance/12943/1 "2013-07-25T16:35:45Z")
Hi, I wanted partial matching functionality on a field so I tried using the nGram tokenizer in my index analyzer but just the standard tokenizer in my search analyzer which worked perfectly. However my question is how the performance scales when used with a large amount of data because I would assume that this will result in a HUGE amount of tokens in the index. Does anybody know if this is actually an issue in elasticsearch, or would the best idea be to not use it on fields with a lot of text (e.g. an article body) but smaller ones (e.g. an article headline). Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [July 26, 2013, 9:25am UTC](https://discuss.elastic.co/t/ngram-performance/12943/2 "2013-07-26T09:25:56Z")
Hi, On Thu, Jul 25, 2013 at 6:35 PM, Michael Lockwood [mlockwood.1989@gmail.com](mailto:mlockwood.1989@gmail.com)wrote: > Hi, > > I wanted partial matching functionality on a field so I tried using the > nGram tokenizer in my index analyzer but just the standard tokenizer in my > search analyzer which worked perfectly. > > However my question is how the performance scales when used with a large > amount of data because I would assume that this will result in a HUGE > amount of tokens in the index. Does anybody know if this is actually an > issue in elasticsearch, or would the best idea be to not use it on fields > with a lot of text (e.g. an article body) but smaller ones (e.g. an article > headline). Indeed, n-grams are going to make your index larger, but if you want to perform partial matching, they are the way to go. Although Lucene/Elasticsearch performs well on prefix queries and there are tricks to make it perform well on suffix queries as well (by applying a filter that reverses the order of the characters), partial matching is very costly as it requires to check every term of the terms dictionary. In that case, n-grams will perform better even if they make the index larger. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 26, 2013, 9:37am UTC](https://discuss.elastic.co/t/ngram-performance/12943/3 "2013-07-26T09:37:43Z")
Just a hint for NGrams if you are after autocompletion. Consider a reasonable configuration. For example, you could check your index size growth for nGrams longer than 2 and smaller than 6. This might reduce some overhead in the dictionary. Jörg On Fri, Jul 26, 2013 at 11:25 AM, Adrien Grand \< [adrien.grand@elasticsearch.com](mailto:adrien.grand@elasticsearch.com)\> wrote: > Hi, > > On Thu, Jul 25, 2013 at 6:35 PM, Michael Lockwood \< > [mlockwood.1989@gmail.com](mailto:mlockwood.1989@gmail.com)\> wrote: > > > Hi, > > > > I wanted partial matching functionality on a field so I tried using the > > nGram tokenizer in my index analyzer but just the standard tokenizer in my > > search analyzer which worked perfectly. > > > > However my question is how the performance scales when used with a large > > amount of data because I would assume that this will result in a HUGE > > amount of tokens in the index. Does anybody know if this is actually an > > issue in elasticsearch, or would the best idea be to not use it on fields > > with a lot of text (e.g. an article body) but smaller ones (e.g. an article > > headline). > > Indeed, n-grams are going to make your index larger, but if you want to > perform partial matching, they are the way to go. Although > Lucene/Elasticsearch performs well on prefix queries and there are tricks > to make it perform well on suffix queries as well (by applying a filter > that reverses the order of the characters), partial matching is very costly > as it requires to check every term of the terms dictionary. In that case, > n-grams will perform better even if they make the index larger. > > -- > Adrien Grand > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:24am UTC](https://discuss.elastic.co/t/ngram-performance/12943/4 "2017-07-06T02:24:23Z")