# IllegalArgumentException: Document contains at least one immense term in field=“abc”.(whose UTF8 encoding is longer than the max length 32766)

**URL:** https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895
**Category:** Elasticsearch
**Created:** [August 14, 2017, 2:48am UTC](https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895 "2017-08-14T02:48:49Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![dimalini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dimalini/32/22448_2.png) [@dimalini](https://discuss.elastic.co/u/dimalini)
#### Post date: [August 14, 2017, 2:48am UTC](https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895/1 "2017-08-14T02:48:49Z")

</div>

I have a field, the mapping of which looks like the following

new **TextProperty**  
{  
Name = "allContent",  
Analyzer = "contentanalyzer",  
**IndexOptions = IndexOptions.Offsets,**  
Norms = false  
}  
The analyzer I am using is a custom one- basically tries to do a regex parsing. And **\_source is enabled** in my case.

During indexing, i get the following exception java.lang.IllegalArgumentException: Document contains at least one immense term in field="allContent" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[48, 120, 52, 100, 53, 97, 57, 48, 48, 48...]...', original message: bytes can be at most 32766 in length; got 627762

I see, the ignore\_above option could have been used had this field been a keyword type. But **the field here is of type text**.

What would be the solution here ?

TIA ~Divya  
[`https://stackoverflow.com/questions/45653094/illegalargumentexception-document-contains-at-least-one-immense-term-in-field`]

---

<div class="post-metadata">

### Author: ![dimalini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dimalini/32/22448_2.png) [@dimalini](https://discuss.elastic.co/u/dimalini)
#### Post date: [August 14, 2017, 5:28am UTC](https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895/2 "2017-08-14T05:28:13Z")

</div>

Basically, it would be helpful if there is some way, where i could say ignore the term that has more than supported byte length.

---

<div class="post-metadata">

### Author: ![rkalhans](https://avatars.discourse-cdn.com/v4/letter/r/7ea924/32.png) [@rkalhans](https://discuss.elastic.co/u/rkalhans)
#### Post date: [August 14, 2017, 5:53am UTC](https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895/3 "2017-08-14T05:53:29Z")

</div>

Hello Divya,

One way to ensure this is that the regex should be created in a way that it  
runs only certain number of iterations. This will be particularly useful  
when working with unknowns content, since unbounded regex matches will lead  
to catastrophic failures like OOM etc. In practice Regex should not run  
unlimited number of times, esp in prod.

Check this link about limiting the matches to a limited length.

> <https://stackoverflow.com/questions/1649435/regular-expression-to-limit-number-of-characters-to-10>

here is an example .

Regex: `((class) {1,3060}(\w{1,10})[]{1,3060})\{`  
matches any content like `class MyClassName {` and groups the keywords and the class Name.

Check explanation here. [https://regex101.com/](https://regex101.com/)  
H2H  
~Rohit

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 11, 2017, 5:53am UTC](https://discuss.elastic.co/t/illegalargumentexception-document-contains-at-least-one-immense-term-in-field-abc-whose-utf8-encoding-is-longer-than-the-max-length-32766/96895/4 "2017-09-11T05:53:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
