# \[FATAL\] Elasticsearch BulkHandler: Unrecoverable indexing failure \[400\]

**URL:** <https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957>\
**Category:** Elasticsearch\
**Created:** [January 11, 2019, 9:37pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957 "2019-01-11T21:37:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![netops](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@netops](https://discuss.elastic.co/u/netops)\
**Post date:** [January 11, 2019, 9:37pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/1 "2019-01-11T21:37:07Z")

</div>

Hello,

We are using Elasticesearch 6.5.4 with SugarCRM Professional version 8.0.2 on site version

Global search is working fine within Sugar, but we are seeing this in the elasticesearch.log

It is showing the same index "ea842a77ff2db255971c5b09f02b0028\_cases" over and over again.

* * *

[root@sugarcrm-01 tmp]# tail -f /var/log/elasticsearch/elasticsearch.log

[2019-01-11T00:00:20,008][DEBUG][o.e.a.b.TransportShardBulkAction] [2e6qEfw] [ea842a77ff2db255971c5b09f02b0028\_cases][2] failed to execute bulk item (index) index {[ea842a77ff2db255971c5b09f02b0028\_cases][Cases][f190e234-e136-3a37-d495-543d6eb545c0], source[n/a, actual length: [35.3kb], max length: 2kb]}  
java.lang.IllegalArgumentException: DocValuesField "description" is too large, must be \<= 32766  
at org.apache.lucene.index.SortedSetDocValuesWriter.addValue(SortedSetDocValuesWriter.java:82) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
at org.apache.lucene.index.DefaultIndexingChain.indexDocValue(DefaultIndexingChain.java:615) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:467) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:394) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494) ~[lucene-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:01:13]  
org.elasticsearch.action.bulk.TransportShardBulkAction.executeOnPrimaryWhileHandlingMappingUpdates(TransportShardBulkAction.java:483) ~[elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.bulk.TransportShardBulkAction.executeIndexRequestOnPrimary(TransportShardBulkAction.java:459) [elasticsearch-6.5.4.jar:6.5.4]  
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:79) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1022) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryShardReference.perform(TransportReplicationAction.java:1000) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.ReplicationOperation.execute(ReplicationOperation.java:102) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:356) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.onResponse(TransportReplicationAction.java:296) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:963) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$1.onResponse(TransportReplicationAction.java:960) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:271) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.index.shard.IndexShardOperationPermits.acquire(IndexShardOperationPermits.java:238) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.index.shard.IndexShard.acquirePrimaryOperationPermit(IndexShard.java:2327) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction.acquirePrimaryShardReference(TransportReplicationAction.java:972) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction.access$500(TransportReplicationAction.java:97) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$AsyncPrimaryAction.doRun(TransportReplicationAction.java:317) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:292) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler$1.doRun(SecurityServerTransportInterceptor.java:251) [x-pack-security-6.5.4.jar:6.5.4]  
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:309) [x-pack-security-6.5.4.jar:6.5.4]  
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:66) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:717) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:723) [elasticsearch-6.5.4.jar:6.5.4]  
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]  
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0\_191]  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0\_191]  
at java.lang.Thread.run(Unknown Source) [?:1.8.0\_191]  
[2019-01-11T00:00:28,694][DEBUG][o.e.a.b.TransportShardBulkAction] [2e6qEfw] [ea842a77ff2db255971c5b09f02b0028\_cases][4] failed to execute bulk item (index) index {[ea842a77ff2db255971c5b09f02b0028\_cases][Cases][af6e7235-7c12-b075-ed1b-543d8ee6cdc1], source[n/a, actual length: [53.5kb], max length: 2kb]}  
java.lang.IllegalArgumentException: DocValuesField "description" is too large, must be \<= 32766

---

<div class="post-metadata">

**Author:** ![netops](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@netops](https://discuss.elastic.co/u/netops)\
**Post date:** [January 11, 2019, 9:40pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/2 "2019-01-11T21:40:14Z")

</div>

[root@sugarcrm-01 tmp]# java -version  
java version "1.8.0\_191"  
Java(TM) SE Runtime Environment (build 1.8.0\_191-b12)  
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)

[root@sugarcrm-01 tmp]# ps -ef | grep elasticsearch  
elastic+ 11797 1 6 Jan10 ? 01:04:35 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch.K9GK8AO7 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs\_err\_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/var/log/elasticsearch/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=rpm -cp /usr/share/elasticsearch/lib/\* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet  
elastic+ 11860 11797 0 Jan10 ? 00:00:00 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86\_64/bin/controller  
root 26704 26086 0 15:39 pts/1 00:00:00 grep --color=auto elasticsearch  
[root@sugarcrm-01 tmp]#

---

<div class="post-metadata">

**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\
**Post date:** [January 11, 2019, 10:28pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/3 "2019-01-11T22:28:37Z")

</div>

1. Please format your code for better readability.
2. The relevant error message is `DocValuesField "description" is too large, must be <= 32766`. The relevant information for this is in the mapping of the description field — please [get that information](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html#indices-get-mapping) and add it. Overall the problem is that the value of the description field is longer than expected (32KB). and you'll either have to fix your data, the way you store that information, or your mapping.

---

<div class="post-metadata">

**Author:** ![netops](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@netops](https://discuss.elastic.co/u/netops)\
**Post date:** [January 11, 2019, 10:42pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/4 "2019-01-11T22:42:23Z")

</div>

Thank you xeraa

Sorry, I am new to this and Elasticsearch..

Please can you provide details on how do I update the mapping in Elasticsearh ?

Thanks again.

---

<div class="post-metadata">

**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\
**Post date:** [January 12, 2019, 1:37am UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/5 "2019-01-12T01:37:18Z")

</div>

Updating mappings is unfortunately not as simple as in relational databases. But first we need the actual mapping. The command should look something like the following, but you'll have to change hostname and port if they are not the defaults:

```
curl -XGET http://localhost:9200/ea842a77ff2db255971c5b09f02b0028_cases/_mapping
```

---

<div class="post-metadata">

**Author:** ![netops](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@netops](https://discuss.elastic.co/u/netops)\
**Post date:** [January 12, 2019, 2:20am UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/6 "2019-01-12T02:20:47Z")

</div>

Thank you again

Here is the info. it display

[root@sugarcrm-01 ~]# curl -XGET [http://localhost:9200/ea842a77ff2db255971c5b09f02b0028\_cases/\_mapping](http://localhost:9200/ea842a77ff2db255971c5b09f02b0028_cases/_mapping)  
{"ea842a77ff2db255971c5b09f02b0028\_cases":{"mappings":{"Cases":{"dynamic":"false","\_all":{"enabled":false},"properties":{"Cases\_\_assigned\_user\_id":{"type":"keyword","index":false,"fields":{"gs\_not\_analyzed":{"type":"keyword","store":true}}},"Cases\_\_case\_number":{"type":"keyword","index":false,"fields":{"gs\_integer":{"type":"integer","index":false},"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"},"gs\_string\_wildcard":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string\_ngram","search\_analyzer":"gs\_analyzer\_string"}}},"Cases\_\_created\_by":{"type":"keyword","index":false,"fields":{"gs\_not\_analyzed":{"type":"keyword","store":true}}},"Cases\_\_date\_entered":{"type":"keyword","index":false,"fields":{"gs\_datetime":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"}}},"Cases\_\_date\_modified":{"type":"keyword","index":false,"fields":{"gs\_datetime":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"}}},"Cases\_\_description":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"}}},"Cases\_\_initial\_contact\_email\_c":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"},"gs\_string\_wildcard":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string\_ngram","search\_analyzer":"gs\_analyzer\_string"}}},"Cases\_\_initial\_contact\_officephone\_c":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"},"gs\_string\_wildcard":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string\_ngram","search\_analyzer":"gs\_analyzer\_string"}}},"Cases\_\_modified\_user\_id":{"type":"keyword","index":false,"fields":{"gs\_not\_analyzed":{"type":"keyword","store":true}}},"Cases\_\_name":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"},"gs\_string\_wildcard":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string\_ngram","search\_analyzer":"gs\_analyzer\_string"}}},"Cases\_\_resolution":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"}}},"Cases\_\_slx\_ticket\_no\_c":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"},"gs\_string\_wildcard":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string\_ngram","search\_analyzer":"gs\_analyzer\_string"}}},"Cases\_\_team\_set\_id":{"type":"keyword","index":false,"fields":{"set":{"type":"keyword"}}},"Cases\_\_work\_log":{"type":"keyword","index":false,"fields":{"gs\_string":{"type":"text","store":true,"analyzer":"gs\_analyzer\_string"}}},"Common\_\_assigned\_user\_id":{"type":"keyword","index":false,"fields":{"agg":{"type":"keyword"}}},"Common\_\_created\_by":{"type":"keyword","index":false,"fields":{"agg":{"type":"keyword"}}},"Common\_\_date\_modified":{"type":"keyword","index":false,"fields":{"gs\_datetime":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"}}},"Common\_\_erased\_fields":{"type":"keyword","index":false,"fields":{"erased\_fields":{"type":"keyword"}}},"Common\_\_favorite\_link":{"type":"keyword","index":false,"fields":{"agg":{"type":"keyword"}}},"Common\_\_modified\_user\_id":{"type":"keyword","index":false,"fields":{"agg":{"type":"keyword"}}},"Common\_\_owner\_id":{"type":"keyword","index":false,"fields":{"owner":{"type":"keyword"}}},"Common\_\_tags":{"type":"keyword","index":false,"fields":{"tags":{"type":"keyword"}}},"Common\_\_user\_favorites":{"type":"keyword","index":false,"fields":{"agg":{"type":"keyword"}}},"assigned\_user\_id":{"type":"keyword","index":false,"copy\_to":["Cases\_\_assigned\_user\_id","Common\_\_assigned\_user\_id"]},"case\_number":{"type":"keyword","index":false,"copy\_to":["Cases\_\_case\_number"]},"created\_by":{"type":"keyword","index":false,"copy\_to":["Cases\_\_created\_by","Common\_\_created\_by"]},"date\_entered":{"type":"keyword","index":false,"copy\_to":["Cases\_\_date\_entered"]},"date\_modified":{"type":"keyword","index":false,"copy\_to":["Cases\_\_date\_modified","Common\_\_date\_modified"]},"description":{"type":"keyword","index":false,"copy\_to":["Cases\_\_description"]},"erased\_fields":{"type":"keyword","index":false,"copy\_to":["Common\_\_erased\_fields"]},"favorite\_link":{"type":"keyword","index":false,"copy\_to":["Common\_\_favorite\_link"]},"initial\_contact\_email\_c":{"type":"keyword","index":false,"copy\_to":["Cases\_\_initial\_contact\_email\_c"]},"initial\_contact\_officephone\_c":{"type":"keyword","index":false,"copy\_to":["Cases\_\_initial\_contact\_officephone\_c"]},"modified\_user\_id":{"type":"keyword","index":false,"copy\_to":["Cases\_\_modified\_user\_id","Common\_\_modified\_user\_id"]},"name":{"type":"keyword","index":false,"copy\_to":["Cases\_\_name"]},"owner\_id":{"type":"keyword","index":false,"copy\_to":["Common\_\_owner\_id"]},"resolution":{"type":"keyword","index":false,"copy\_to":["Cases\_\_resolution"]},"slx\_ticket\_no\_c":{"type":"keyword","index":false,"copy\_to":["Cases\_\_slx\_ticket\_no\_c"]},"tags":{"type":"keyword","index":false,"copy\_to":["Common\_\_tags"]},"team\_set\_id":{"type":"keyword","index":false,"copy\_to":["Cases\_\_team\_set\_id"]},"user\_favorites":{"type":"keyword","index":false,"copy\_to":["Common\_\_user\_favorites"]},"work\_log":{"type":"keyword","index":false,"copy\_to":["Cases\_\_work\_log"]}}}}}}

---

<div class="post-metadata">

**Author:** ![netops](https://avatars.discourse-cdn.com/v4/letter/n/54ee81/32.png) [@netops](https://discuss.elastic.co/u/netops)\
**Post date:** [January 14, 2019, 10:17pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/7 "2019-01-14T22:17:38Z")

</div>

Hello xeraa,

Any update please ?

I am not sure how to update the mapping properly with curl -XPUT [http://localhost:9200/](http://localhost:9200/).......

I also came across this link when researching this issue.

> <https://stackoverflow.com/questions/24019868/utf8-encoding-is-longer-than-the-max-length-32766>

Here is an example of how I use it in php:

```auto
'mapping' => [
    'type' => 'multi_field',
    'path' => 'full',
    'fields' => [
        '{name}' => [
            'type' => 'string',
            'index' => 'analyzed',
            'analyzer' => 'standard',
        ],
        'raw' => [
            'type' => 'string',
            'index' => 'not_analyzed',
            'ignore_above' => 256,
        ],
    ],
],

```

In your case you probably want to do as John Petrone told you and set `"index": "no"` but for anyone else finding this question after, like me, searching on that Exception then your options are:

- set `"index": "no"`
- set `"index": "analyze"`
- set `"index": "not_analyzed"` and `"ignore_above": 256`

---

<div class="post-metadata">

**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\
**Post date:** [January 15, 2019, 12:22pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/8 "2019-01-15T12:22:38Z")

</div>

Yes, those 3 are your options. The main question is why description would be a keyword / not analyzed? That's normally helpful for sorting by that field or running aggregations on it. Whereas analyzed fields are for full-text search. I'd assume that description should be searchable, so you'd want it to be analyzed.

Unfortunately you can't change that mapping after creation. If you're still prototyping, delete the data and change your mapping. If you have production data you'll want to create a new index with the right mapping and `_reindex` your existing data into the new index.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 12, 2019, 12:22pm UTC](https://discuss.elastic.co/t/fatal-elasticsearch-bulkhandler-unrecoverable-indexing-failure-400/163957/9 "2019-02-12T12:22:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.