# Error "String length exceeds the maximum length (5000000)" when transferring a large document to the attachment pipeline **URL:** \ **Category:** Elasticsearch\ **Created:** [October 25, 2023, 3:25am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687 "2023-10-25T03:25:31Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Vlad\_I](https://avatars.discourse-cdn.com/v4/letter/v/dec6dc/32.png) [@Vlad\_I](https://discuss.elastic.co/u/Vlad_I)\ **Post date:** [October 25, 2023, 3:25am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/1 "2023-10-25T03:25:31Z")
I'm using Elasticsearch 8.9.1 Using python, I send an 8MB xlsx document to the Elasticsearch index via attachment pipeline. But the error "String length (5046272) exceeds the maximum length (5000000)" appears. For example, documents with a size of 1MB do not cause an error. How can I increase the maximum string length or otherwise avoid this error? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 7, 2023, 12:38pm UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/2 "2023-11-07T12:38:07Z")
Welcome! Could you share the full stack trace please? That's one of the reason I prefer doing the extraction on the local machine before sending the data to Elasticsearch. This is what FSCrawler is doing. > But the error "String length (5046272) exceeds the maximum length (5000000)" appears. The default limit of 5mb have been increased to 20mb. See: > > > (note: possible follow-up to #959) > > Based on user/dev feedback wrt 2.15.0 rele…ase, there seem to be use cases where JSON String values (note: NOT Document size) exceed 5 million characters. While this is configurable limit it seems best to try to change default to be yet more lenient, given that this is not the likeliest way to try to cause DoS. It is also the case that those who do wish to use more modest sizes will anyway need to lower limits, regardless of defaults we choose. > > My initial thinking is to increase this further to 20 million (giving max memory usage of about 40 megs in JVM -- although temporarily up to twice that due to segmented storage), for Jackson 2.15.1. It's part of Jackson 2.15.1. But AFAICS 2.15.1 is not yet part of 8.10.4... Although it is supposed to be 2gb.... > ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 7, 2023, 12:53pm UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/3 "2023-11-07T12:53:07Z")
I added a comment here: > > > Jackson 2.15 introduced a (rough) maximum limit on string length. This commit re…laxes that limit to its maximum size, leaving document size constraints to other existing limits in the system. We can revisit whether string length within a document should be independently constrained later. > > Fixes #96036 ---
**Author:** ![Vlad\_I](https://avatars.discourse-cdn.com/v4/letter/v/dec6dc/32.png) [@Vlad\_I](https://discuss.elastic.co/u/Vlad_I)\ **Post date:** [November 8, 2023, 2:37am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/4 "2023-11-08T02:37:28Z")
> [@dadoonet](#): > > full stack trace The full error message is below. I don't know how to get full stack trace. {'root\_cause': [{'type': 'document\_parsing\_exception', 'reason': "[-1:21] failed to parse field [attachment.content] of type [text] in document with id 'X5mWECsLu1Sp\_VdsN7Bcng=='. Could not parse field value preview,"}], 'type': 'document\_parsing\_exception', 'reason': "[-1:21] failed to parse field [attachment.content] of type [text] in document with id 'X5mWECsLu1Sp\_VdsN7Bcng=='. Could not parse field value preview,", 'caused\_by': {'type': 'stream\_constraints\_exception', 'reason': 'String length (5046272) exceeds the maximum length (5000000)'}} To avoid the problem described, I'm using Elasticsearch 8.7.1. This version handles 5MB documents successfully, but 44MB documents cause the error below. {'root\_cause': [{'type': 'parse\_exception', 'reason': 'Error parsing document in field [data]'}], 'type': 'parse\_exception', 'reason': 'Error parsing document in field [data]', 'caused\_by': {'type': 'tika\_exception', 'reason': 'Unexpected RuntimeException from org.apache.tika.parser.ParserDecorator$2@5c3c5810', 'caused\_by': {'type': 'record\_format\_exception', 'reason': 'Tried to allocate an array of length 108,463,468, but the maximum length for this record type is 100,000,000.\nIf the file is not corrupt and not large, please open an issue on bugzilla to request \nincreasing the maximum allowable size for this record type.\nYou can set a higher override value with IOUtils.setByteArrayMaxOverride()'}}} Since I need to index very large MS Word/Excel documents, I would like to solve this problem too. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 8, 2023, 3:04am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/5 "2023-11-08T03:04:20Z")
> [@Vlad\_I](#): > > Since I need to index very large MS Word/Excel documents, I would like to solve this problem too. You can't do that with the ingest attachment plugin IMO. But good news, we just released this in 8.11: [Content extraction | Enterprise Search documentation [8.11] | Elastic](https://www.elastic.co/guide/en/enterprise-search/current/connectors-content-extraction.html#connectors-content-extraction-local) I did not test it yet. And I don't know how you can directly use this service if possible. If it still does not work for you, may be give a try to FSCrawler. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [November 8, 2023, 3:25am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/6 "2023-11-08T03:25:44Z")
> [@Vlad\_I](#): > > I don't know how to get full stack trace. It should be in Elasticsearch logs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 6, 2023, 3:26am UTC](https://discuss.elastic.co/t/error-string-length-exceeds-the-maximum-length-5000000-when-transferring-a-large-document-to-the-attachment-pipeline/345687/7 "2023-12-06T03:26:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.