# Elasticsearch Bulk upload issue

**URL:** <https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574>\
**Category:** Elasticsearch\
**Created:** [March 16, 2016, 3:08pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574 "2016-03-16T15:08:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rahul](https://avatars.discourse-cdn.com/v4/letter/r/f14d63/32.png) [@Rahul](https://discuss.elastic.co/u/Rahul)\
**Post date:** [March 16, 2016, 3:08pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574/1 "2016-03-16T15:08:39Z")

</div>

We are facing one issue for the bulk load API we are using for  
ES version 2.1.1. This seems to be issue for the case when automatic index  
creation takes place in ES. It look like, when all those concurrent writes in  
bulk load competes to be the first to create the index, all but one fails. We  
want indices to be automatically created day wise based on template defined in  
ES. Once the index is created we don’t observe this issue for the existing  
indices or even with more data count in bulk load than the one failed with this  
error.

Following is the error observed in this case -

RemoteTransportException[[host][host:ip][indices:admin/create]];  
nested: ProcessClusterEventTimeoutException[failed to process cluster event  
(acquire index lock) within 4s];,index\_2016.03.14

Details:

We are using 3 node cluster for ES where number\_of\_replicas: 2  
and number\_of\_shards: 1 (3 master and 3 data nodes)

We have kept 4 sec as the timeout value for the bulk load which  
we can change (increase) if required.

Is the cause of error is automatic index creation or  
something else and if this is the existing issue, is there any workaround ?

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [March 16, 2016, 4:48pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574/2 "2016-03-16T16:48:19Z")

</div>

The alternative is to create an index explicitly, with all settings and mappings.

---

<div class="post-metadata">

**Author:** ![Rahul](https://avatars.discourse-cdn.com/v4/letter/r/f14d63/32.png) [@Rahul](https://discuss.elastic.co/u/Rahul)\
**Post date:** [March 17, 2016, 2:50pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574/3 "2016-03-17T14:50:27Z")

</div>

Actually we are creating index dynamically, we would like to go with dynamic creation only day wise. Otherwise we will have to write separate program to create those indices everyday!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:07pm UTC](https://discuss.elastic.co/t/elasticsearch-bulk-upload-issue/44574/4 "2017-07-05T23:07:34Z")

</div>