# Max length allowed for "max\_token\_length" and how to set value

**URL:** https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812
**Category:** Elasticsearch
**Created:** [July 31, 2016, 4:15am UTC](https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812 "2016-07-31T04:15:47Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Eli\_Mintz](https://avatars.discourse-cdn.com/v4/letter/e/91b2a8/32.png) [@Eli\_Mintz](https://discuss.elastic.co/u/Eli_Mintz)
#### Post date: [July 31, 2016, 4:15am UTC](https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812/1 "2016-07-31T04:15:47Z")

</div>

Hi,

I would like to set the standard tokenizer to use a token length of 30000 for a field (I am inserting biological sequences into this field) instead of cutting up the string into 255 length tokens as is the default. Can this be done? What is the JSON command to do this?

Thanks.

---

<div class="post-metadata">

### Author: ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)
#### Post date: [July 31, 2016, 8:12am UTC](https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812/2 "2016-07-31T08:12:07Z")

</div>

There is no such JSON command. Expanding token lengths over the default value have massive impact on indexing and search - the default token length is 32k.

What are you trying to do? Search for sequence alignments? If so, there are better solutions: imagine locality sensitive hashes [http://de.slideshare.net/ZacharyTong/boston-meetupgoingorganic/60-hashdistributes\_similar\_values\_into\_unique](http://de.slideshare.net/ZacharyTong/boston-meetupgoingorganic/60-hashdistributes_similar_values_into_unique)

[https://issues.apache.org/jira/browse/LUCENE-6968](https://issues.apache.org/jira/browse/LUCENE-6968) would have to be backported to Elasticsearch 2.3+, in a plugin maybe.

---

<div class="post-metadata">

### Author: ![taras](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/taras/32/507_2.png) [@taras](https://discuss.elastic.co/u/taras)
#### Post date: [August 1, 2016, 1:35am UTC](https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812/3 "2016-08-01T01:35:25Z")

</div>

Another approach to consider is to split up the token yourself into smaller chains of importance, and then use a phrase search.

If you really do want only exact matches, you could insert it as a [not\_analyzed](https://www.elastic.co/guide/en/elasticsearch/reference/current/string.html) string.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:31pm UTC](https://discuss.elastic.co/t/max-length-allowed-for-max-token-length-and-how-to-set-value/56812/4 "2017-07-05T22:31:25Z")

</div>


