I have an index that contains the text of files. Most files aren't that large, but some do have more than 1,000,000 characters. I'm using the default unified highlighter when displaying results to the end users. When a search term that returns one of the large files is executed, I'm getting the following error:
The length of [fileText.stemmed] field of  doc of [attachments] index has exceeded  - maximum allowed to be analyzed for highlighting. This maximum can be set by changing the [index.highlight.max_analyzed_offset] index level setting. For large texts, indexing with offsets or term vectors is recommended! (illegal_argument_exception)
I would prefer to not increase the highlight.max_analyzed_offset setting because it seems like it would hurt performance and I would have to know the max size of the documents in my index which is constantly changing. The error message indicates that "indexing with offsets or term vectors is recommended", but I can not find any documentation on how to implement this. Any pointers on how to handle this would be greatly appreciated.