# Immense term and maximum length exception in elasticsearch output

**URL:** <https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536>\
**Category:** Logstash\
**Created:** [March 14, 2017, 2:27pm UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536 "2017-03-14T14:27:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![fritzhardy](https://avatars.discourse-cdn.com/v4/letter/f/eada6e/32.png) [@fritzhardy](https://discuss.elastic.co/u/fritzhardy)\
**Post date:** [March 14, 2017, 2:27pm UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536/1 "2017-03-14T14:27:50Z")

</div>

I recently began running into this:

"max\_bytes\_length\_exceeded\_exception: bytes can be at most 32766 in length; got 338076"

Full error:

```
{"type"=>"illegal_argument_exception", "reason"=>"Document contains at least one immense term in field=\"some.field.keyword\" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '...', original message: bytes can be at most 32766 in length; got 338076", "caused_by"={"type"=>"max_bytes_length_exceeded_exception", "reason"=>"max_bytes_length_exceeded_exception: bytes can be at most 32766 in length; got 338076"}

```

This is on an ES cluster fairly recently upgraded to 5.x. I know that mapping changes to the new keyword type are reflected in the elasticsearch template that logstash installs, but found it curious that the ignore\_above directive is no longer set.

elastic-logstash-template-es2x.json:

```
"string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" },
            "fields" : {
              "raw" : {"type": "string", "index" : "not_analyzed", "doc_values" : true, "ignore_above" : 256}
            }
          }
        }

```

elastic-logstash-template-es5x.json:

```
"string_fields" : {
  "match" : "*",
  "match_mapping_type" : "string",
  "mapping" : {
    "type" : "text", "norms" : false,
    "fields" : {
      "keyword" : { "type": "keyword" }
    }
  }
}

```

I am now injecting my own template with the following change, which is more analogous to how things were working: fields properly mapped, no .keyword sub-field for items larger than 256 bytes, and therefore no aggregation on them (which is probably not desired anyway)

```
--- elastic-logstash-template-es5x.json	2017-01-24 20:14:18.000000000 +0000
+++ elastic-logstash-template-es5x-fixed.json	2017-03-14 13:48:51.542094141 +0000
@@ -23,7 +23,7 @@
           "mapping" : {
             "type" : "text", "norms" : false,
             "fields" : {
- "keyword" : { "type": "keyword" }
+ "keyword" : { "type": "keyword", "ignore_above": 256 }
             }
           }
         }

```

Was this an oversight, or a purposeful change? It seems to me this should still be default behavior.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 11, 2017, 2:28pm UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536/2 "2017-04-11T14:28:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 9, 2018, 3:39pm UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536/3 "2018-04-09T15:39:45Z")

</div>



---

<div class="post-metadata">

**Author:** ![fritzhardy](https://avatars.discourse-cdn.com/v4/letter/f/eada6e/32.png) [@fritzhardy](https://discuss.elastic.co/u/fritzhardy)\
**Post date:** [April 9, 2018, 8:10pm UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536/4 "2018-04-09T20:10:16Z")

</div>

This was ultimately tracked in [https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/588](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/588), and fixed with [https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/610](https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/610), which did essentially the above.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 4:41am UTC](https://discuss.elastic.co/t/immense-term-and-maximum-length-exception-in-elasticsearch-output/78536/5 "2022-11-04T04:41:37Z")

</div>