# Timeout during bulk indexing when doc size increased

**URL:** <https://discuss.elastic.co/t/timeout-during-bulk-indexing-when-doc-size-increased/6733>\
**Category:** Elasticsearch\
**Created:** [February 16, 2012, 6:33pm UTC](https://discuss.elastic.co/t/timeout-during-bulk-indexing-when-doc-size-increased/6733 "2012-02-16T18:33:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![dragan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dragan/32/2943_2.png) [@dragan](https://discuss.elastic.co/u/dragan)\
**Post date:** [February 16, 2012, 6:33pm UTC](https://discuss.elastic.co/t/timeout-during-bulk-indexing-when-doc-size-increased/6733/1 "2012-02-16T18:33:17Z")

</div>

Hi,

I have the following setup:

Two ES nodes on ec2 (fedora) with 5 shards 1 replica each, default  
elasticsearch.conf, a separate ec2 instance (ubuntu 10.10) that  
indexes documents to ES via pyes 0.16

I'm indexing documents in batches of less than 5 thousand docs.

If each document is less than 2Kb, everything runs smoothly.  
If I populate a string field that has a paragraph of text in it,  
bringing the size of a doc to ~4Kb, the insertion process times out.

The timeout I set in pyes when I create the connection object is 2  
minutes, my input files are getting larger and I will need to be able  
to push more data per document into ES. I'm wondering what is causing  
the timeout and what I need to change in the config in order to push  
into the ES reliably.

If you can share some rules of thumb, I'd greatly appreciate that  
Thank you

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [February 16, 2012, 8:21pm UTC](https://discuss.elastic.co/t/timeout-during-bulk-indexing-when-doc-size-increased/6733/2 "2012-02-16T20:21:24Z")

</div>

First, depending on your instance type, I would suggest changing the default memory allocation to explicitly set the memory set for the ES (java) process.

I don't know what times out and why, did you see any failures in elasticsearch logs? If you increase the time out value, does it work?

On Thursday, February 16, 2012 at 8:33 PM, Dragan wrote:

> Hi,
> 
> I have the following setup:
> 
> Two ES nodes on ec2 (fedora) with 5 shards 1 replica each, default  
> elasticsearch.conf, a separate ec2 instance (ubuntu 10.10) that  
> indexes documents to ES via pyes 0.16
> 
> I'm indexing documents in batches of less than 5 thousand docs.
> 
> If each document is less than 2Kb, everything runs smoothly.  
> If I populate a string field that has a paragraph of text in it,  
> bringing the size of a doc to ~4Kb, the insertion process times out.
> 
> The timeout I set in pyes when I create the connection object is 2  
> minutes, my input files are getting larger and I will need to be able  
> to push more data per document into ES. I'm wondering what is causing  
> the timeout and what I need to change in the config in order to push  
> into the ES reliably.
> 
> If you can share some rules of thumb, I'd greatly appreciate that  
> Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:39am UTC](https://discuss.elastic.co/t/timeout-during-bulk-indexing-when-doc-size-increased/6733/3 "2017-07-06T03:39:01Z")

</div>