# Indexing a large pdf file (around 90MB) gives an exception

**URL:** <https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014>\
**Category:** Elasticsearch\
**Created:** [January 18, 2018, 10:05am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014 "2018-01-18T10:05:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Abhilash\_Bolla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhilash_bolla/32/26840_2.png) [@Abhilash\_Bolla](https://discuss.elastic.co/u/Abhilash_Bolla)\
**Post date:** [January 18, 2018, 10:05am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014/1 "2018-01-18T10:05:11Z")

</div>

Hi,

Elasticsearch version: 5.4.3  
Heap allocated: 4GB  
Java version: Java 8  
I am trying to index a large pdf file of 90 MB using the ingest-attachments plugin. I encode the pdf file into base64 format and then index it by calling the .index() function of the python client. I get this error:

```auto
ConnectionError(('Connection aborted.', error(104, 'Connection reset by peer'))) caused by: ProtocolError(('Connection aborted.', error(104, 'Connection reset by peer')))

```

I am successfully able to index smaller files but the process gets terminated when I encounter this file. Any help is appreciated. Thanks.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 18, 2018, 10:21am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014/2 "2018-01-18T10:21:41Z")

</div>

Some comments:

- ingest-mapper-attachments? Is it ingest-attachment or mapper-attachments plugin? I guess the former.
- pdf file of 90 MB: which means probably more in term of JSON as you encode it using a BASE64. What is the size of the JSON doc?
- Have a look at `http.max_content_length` on [https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html). By default, it's limited to 100mb

If you intend to send very big documents like this, may be `4GB` of HEAP is going to be too small at some point. Monitor it to make sure you are not putting too much pressure.

You can also use an external process to do the document extraction, like FSCrawler or using Tika directly.

---

<div class="post-metadata">

**Author:** ![Abhilash\_Bolla](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhilash_bolla/32/26840_2.png) [@Abhilash\_Bolla](https://discuss.elastic.co/u/Abhilash_Bolla)\
**Post date:** [January 18, 2018, 10:27am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014/3 "2018-01-18T10:27:29Z")

</div>

My bad, its ingest-attachment plugin.

As per the point 3 of your comments, I guess the python client is getting timed out. I don't see anything in Elasticsearch logs regarding "http.max\_content\_length" or even the GCs kicking in if 4GB heap is too small.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 18, 2018, 10:45am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014/4 "2018-01-18T10:45:30Z")

</div>

> I guess the python client is getting timed out.

Might be indeed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 15, 2018, 10:45am UTC](https://discuss.elastic.co/t/indexing-a-large-pdf-file-around-90mb-gives-an-exception/116014/5 "2018-02-15T10:45:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.