I have a field, the mapping of which looks like the following
Name = "allContent",
Analyzer = "contentanalyzer",
IndexOptions = IndexOptions.Offsets,
Norms = false
The analyzer I am using is a custom one- basically tries to do a regex parsing. And _source is enabled in my case.
During indexing, i get the following exception java.lang.IllegalArgumentException: Document contains at least one immense term in field="allContent" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[48, 120, 52, 100, 53, 97, 57, 48, 48, 48...]...', original message: bytes can be at most 32766 in length; got 627762
I see, the ignore_above option could have been used had this field been a keyword type. But the field here is of type text.
What would be the solution here ?