# Limit the max document size after ingest pipeline **URL:** \ **Category:** Elasticsearch\ **Tags:** ingest-pipeline\ **Created:** [March 2, 2021, 2:33am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892 "2021-03-02T02:33:28Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Crazedpeanut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crazedpeanut/32/84779_2.png) [@Crazedpeanut](https://discuss.elastic.co/u/Crazedpeanut)\ **Post date:** [March 2, 2021, 2:33am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/1 "2021-03-02T02:33:28Z")
Is there way way to limit the size of a document after an ingest pipeline has processed it? For example, I want to limit the size of indexed documents to 1mb. A 4mb PDF might be uploaded, the attachment processor extracts the text, the original PDF data is removed, now the size of the document to be indexed should be under the maximum 1mb max\_content\_length seems to limit the upload of the document prior to the pipeline. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 2, 2021, 2:54am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/2 "2021-03-02T02:54:27Z")
Welcome to our community! 😃 You might be able to do this with a script, but I haven't see it done nor how to do it myself. Is there a reason you want to do this? ---
**Author:** ![Crazedpeanut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crazedpeanut/32/84779_2.png) [@Crazedpeanut](https://discuss.elastic.co/u/Crazedpeanut)\ **Post date:** [March 2, 2021, 3:26am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/3 "2021-03-02T03:26:51Z")
Thanks for the welcome 🙂 The system we are building is to ingest lots of documents that might range in size. Normally, the size of the document is quite small after the pipeline has finished. Occasionally, this is not the case so we then have a large document in Elastic that causes some issues (slow, unresponsive, etc) for our services when they come across it. The tricky part is that we don't know what the size of the document is until it has been processed. I guess we can add a script to check the size of the attachment processor result. I was just wondering what other options might exist. There is no setting at the index level for maximum document size? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 2, 2021, 3:54am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/4 "2021-03-02T03:54:32Z")
Have a look at `indexed_chars` option. > **[Using the Attachment Processor in a Pipeline | Elasticsearch Plugins and...](https://www.elastic.co/guide/en/elasticsearch/plugins/7.11/using-ingest-attachment.html)** It limits the number of extracted chars when running the attachment plugin. HTH ---
**Author:** ![Crazedpeanut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crazedpeanut/32/84779_2.png) [@Crazedpeanut](https://discuss.elastic.co/u/Crazedpeanut)\ **Post date:** [March 2, 2021, 4:13am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/5 "2021-03-02T04:13:07Z")
Thanks David, I think this might limit the size of the incoming payload, rather than the output of an ingest pipeline? ---
**Author:** ![Crazedpeanut](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crazedpeanut/32/84779_2.png) [@Crazedpeanut](https://discuss.elastic.co/u/Crazedpeanut)\ **Post date:** [March 2, 2021, 4:20am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/6 "2021-03-02T04:20:10Z")
I have this processor stage, which drops the document if the output is over 512kb. I guess I was after a policy that could be applied on an index or multiple indices, rather than at the pipeline level. `{ "drop": { "if": "!ctx.containsKey('attachment') || !ctx['attachment'].containsKey('content') || ctx['attachment']['content_length'] > 512000" } }` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [March 2, 2021, 4:32am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/7 "2021-03-02T04:32:05Z")
Nope. That limits the number of extracted characters whatever the size of the input. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 30, 2021, 4:32am UTC](https://discuss.elastic.co/t/limit-the-max-document-size-after-ingest-pipeline/265892/8 "2021-03-30T04:32:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.