# Does EdgeNGram Tokenizer internally optimize for a finite set of values regardless of document count?

**URL:** <https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478>\
**Category:** Elasticsearch\
**Created:** [August 9, 2017, 4:47pm UTC](https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478 "2017-08-09T16:47:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nithin\_Chandy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithin_chandy/32/31154_2.png) [@Nithin\_Chandy](https://discuss.elastic.co/u/Nithin_Chandy)\
**Post date:** [August 9, 2017, 4:47pm UTC](https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478/1 "2017-08-09T16:47:02Z")

</div>

If there are only few possible values for a field, lets say max 20 possible values for a field, is it a good idea to use edgeNgram tokenizer for autocompletion?

Just curious if EdgeNGram Tokenizer has any advantage over other tokenizers especially if the possible values are finite.

I assume since the values are finite, it won't consume that much disk space for ngrams regardless of the number of documents and can easily fit in memory (assuming limited number of fields in each doc). Is this true?

---

<div class="post-metadata">

**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\
**Post date:** [August 11, 2017, 8:06am UTC](https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478/2 "2017-08-11T08:06:47Z")

</div>

If there are only 20 values in total, I would probably use a `prefix` query directly, since the `prefix` query will be rewritten to a `bool` query that has 20 terms at most, which is very reasonable.

Your assumption about the cost of edge ngrams being lower when the number of unique values is contained is correct.

---

<div class="post-metadata">

**Author:** ![Nithin\_Chandy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nithin_chandy/32/31154_2.png) [@Nithin\_Chandy](https://discuss.elastic.co/u/Nithin_Chandy)\
**Post date:** [August 18, 2017, 9:15pm UTC](https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478/3 "2017-08-18T21:15:58Z")

</div>

Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2017, 9:16pm UTC](https://discuss.elastic.co/t/does-edgengram-tokenizer-internally-optimize-for-a-finite-set-of-values-regardless-of-document-count/96478/4 "2017-09-15T21:16:01Z")

</div>

Closed on September 15, 2017, 9:16pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.