# Annotated text plugin and index\_options/term\_vectors

**URL:** <https://discuss.elastic.co/t/annotated-text-plugin-and-index-options-term-vectors/263989>\
**Category:** Elasticsearch\
**Created:** [February 11, 2021, 10:33am UTC](https://discuss.elastic.co/t/annotated-text-plugin-and-index-options-term-vectors/263989 "2021-02-11T10:33:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![robmartin11](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robmartin11/32/99177_2.png) [@robmartin11](https://discuss.elastic.co/u/robmartin11)\
**Post date:** [February 11, 2021, 10:33am UTC](https://discuss.elastic.co/t/annotated-text-plugin-and-index-options-term-vectors/263989/1 "2021-02-11T10:33:32Z")

</div>

Im using the annotated\_text field to index documents but for larger documents Im getting errors:  
`index has exceeded [1000000] - maximum allowed to be analyzed for highlighting. This maximum can be set by changing the [index.highlight.max_analyzed_offset] index level setting. For large texts, indexing with offsets or term vectors is recommended!`.

Ive tried adding `"index_options": "offsets"` and `"term_vector": "with_positions_offsets"` but neither seem to have any effect:

```auto
PUT test_index
{
    "mappings": {
        "properties": {
            "annotated_with_offsets": {
                "type": "annotated_text",
                "index_options": "offsets"
            },
            "annotated_with_term_vector": {
                "type": "annotated_text",
                "term_vector": "with_positions_offsets"
            },
            "text_with_offsets": {
                "type": "text",
                "index_options": "offsets"
            },
            "text_with_term_vector": {
                "type": "text",
                "term_vector": "with_positions_offsets"
            }
        }
    }
}

```

```auto
GET /test_index/_mapping

{
    "test_index": {
        "mappings": {
            "properties": {
                "annotated_with_offsets": {
                    "type": "annotated_text"
                },
                "annotated_with_term_vector": {
                    "type": "annotated_text"
                },
                "text_with_offsets": {
                    "type": "text",
                    "index_options": "offsets"
                },
                "text_with_term_vector": {
                    "type": "text",
                    "term_vector": "with_positions_offsets"
                }
            }
        }
    }
}

```

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [February 15, 2021, 9:39am UTC](https://discuss.elastic.co/t/annotated-text-plugin-and-index-options-term-vectors/263989/2 "2021-02-15T09:39:39Z")

</div>

This error is a safeguard kicking in to avoid servers being overloaded by expensive highlighting tasks.  
There is some [work underway](https://github.com/elastic/elasticsearch/pull/67325) to avoid these errors failing searches and just curtail any highlighting work on individual long docs.

The annotated text field has a custom highlighter that does some special tricks to deal with annotations which sadly means it doesn't use term\_vectors so that won't help here. Some potential workarounds:

- You could try increase the `index.highlight.max_analyzed_offset` setting (I ended up doing that to look at Wikipedia content).
- You could run a query with a `must_not` on the size of the doc to avoid trying to highlight super-large documents and hitting this error.
- You could break up large documents into smaller ones e.g. instead of indexing whole books, index chapters.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 15, 2021, 9:39am UTC](https://discuss.elastic.co/t/annotated-text-plugin-and-index-options-term-vectors/263989/3 "2021-03-15T09:39:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.