# Ignore\_above for text datatype in elasticsearch

**URL:** <https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606>\
**Category:** Elasticsearch\
**Created:** [August 17, 2019, 4:06pm UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606 "2019-08-17T16:06:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [August 17, 2019, 4:06pm UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/1 "2019-08-17T16:06:47Z")

</div>

In elasticsearch 5 `string` datatype has been removed as explained nicely in this [blog](https://www.elastic.co/blog/strings-are-dead-long-live-strings) post.

`ignore_above` is not supported with `text` datatype. Can someone let me know how does `text` field is then protected against Lucene’s term byte-length limit as explained in the documentation [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/ignore-above.html), especially the below section

> This option is also useful for protecting against Lucene’s term byte-length limit of `32766` .

---

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [August 20, 2019, 9:46am UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/2 "2019-08-20T09:46:18Z")

</div>

Anyone any thoughts on this? Same has been asked in [stackoverflow](https://stackoverflow.com/questions/57569850/ignore-above-for-text-datatype-in-elasticsearch) also and no response there also.

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [August 20, 2019, 10:01am UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/3 "2019-08-20T10:01:57Z")

</div>

> [@Deb](#):
>
> how does `text` field is then protected against Lucene’s term byte-length limit

Maybe by using the [token length filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-length-tokenfilter.html) in an analyzer config?

---

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [August 20, 2019, 10:30am UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/4 "2019-08-20T10:30:54Z")

</div>

Thanks @Mark_Harwood for replying.

So this I have to specify in analyzer config or by default elasticsearch does it for text field ?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [August 20, 2019, 10:33am UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/5 "2019-08-20T10:33:59Z")

</div>

It's not on by default.  
I've not generally had a need for it because most text gets sliced up by tokenizers into smaller tokens anyway based on whitespace, punctuation etc.  
Maybe a base64 encoded image might produce a single big token but I've not ran into content like that

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 17, 2019, 10:34am UTC](https://discuss.elastic.co/t/ignore-above-for-text-datatype-in-elasticsearch/195606/6 "2019-09-17T10:34:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
