# Bulk insertion taking long and throwing lots of errors

**URL:** <https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390>\
**Category:** Elasticsearch\
**Created:** [July 11, 2012, 7:52pm UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390 "2012-07-11T19:52:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vinicius\_Carvalho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinicius_carvalho/32/770_2.png) [@Vinicius\_Carvalho](https://discuss.elastic.co/u/Vinicius_Carvalho)\
**Post date:** [July 11, 2012, 7:52pm UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/1 "2012-07-11T19:52:53Z")

</div>

Hi there. Well, getting my feet wet with ES. I'm doing a small poc here,  
and I created an index with default shard/replica values (did not touch  
anything). The only thing I changed was the analyzer:

"albums" : {  
"settings" : {  
"index.analysis.analyzer.text\_en.filter.2" : "porterStem",  
"index.analysis.analyzer.text\_en.filter.1" : "lowercase",  
"index.analysis.analyzer.text\_en.tokenizer" : "standard",  
"index.analysis.analyzer.text\_en.type" : "custom",  
"index.analysis.analyzer.text\_en.filter.0" : "standard",  
"index.number\_of\_shards" : "5",  
"index.number\_of\_replicas" : "1",  
"index.version.created" : "190899"  
}  
}

So I'm trying to index a small amount of documents using the BulkRequest,  
but it's taking forever (almost 1 min). And when it finishes it fails most  
of requests, I got only 376 docs indexed out of 1000, and a lot  
of UnavailableShardsException[[albums][2] [2] shardIt, [0] active : Timeout  
waiting for [1m], request:  
org.elasticsearch.action.bulk.BulkShardRequest@db766c1].

I would imagine this is related to the shard config. I'm only running using  
a single instance for now.

The code for the bulk load:

while(rs.next() && count \< max){

count++;

XContentBuilder content = createContent(rs);

bulkRequest.add(client.prepareIndex("albums", "album"  
).setSource(content));

if(count%1000 == 0){

System.out.println("commit");

BulkResponse bulkResponse = bulkRequest.execute().actionGet();

if(bulkResponse.hasFailures()){

for(BulkItemResponse r : bulkResponse.items()){

System.out.println(r.getFailureMessage());

}

System.out.println("ERROR");

```
  }

```

}

}

How can I make it faster for a single node POC?

Regards

---

<div class="post-metadata">

**Author:** ![shoebmogal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shoebmogal/32/2814_2.png) [@shoebmogal](https://discuss.elastic.co/u/shoebmogal)\
**Post date:** [July 11, 2012, 8:33pm UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/2 "2012-07-11T20:33:39Z")

</div>

After every bulk insert, re-initializing the bulkRequest variable:

bRequest = client.prepareBulk();

Set refresh rate = -1  
Set no of replicas to 0

Let me know the results!

Thanks,  
Shoeb 🙂

---

<div class="post-metadata">

**Author:** ![Vinicius\_Carvalho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinicius_carvalho/32/770_2.png) [@Vinicius\_Carvalho](https://discuss.elastic.co/u/Vinicius_Carvalho)\
**Post date:** [July 11, 2012, 8:55pm UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/3 "2012-07-11T20:55:16Z")

</div>

Hi! fixed this by changing the shard/replica values. Still a bit slower  
when compared to Solr DataImporthandler

Besides that, tons of errors on duplicateddocuments are arising, which  
makes no sense, as I'm not sending duplicate documents. Only thing I could  
think is that ids are being duplicated (I'm not sending ids, instead, I'm  
relying on automatic generation of id)

Still not getting why.

Any ideas?

REgards

On Wednesday, July 11, 2012 3:52:53 PM UTC-4, Vinicius Carvalho wrote:

> Hi there. Well, getting my feet wet with ES. I'm doing a small poc here,  
> and I created an index with default shard/replica values (did not touch  
> anything). The only thing I changed was the analyzer:
> 
> "albums" : {  
> "settings" : {  
> "index.analysis.analyzer.text\_en.filter.2" : "porterStem",  
> "index.analysis.analyzer.text\_en.filter.1" : "lowercase",  
> "index.analysis.analyzer.text\_en.tokenizer" : "standard",  
> "index.analysis.analyzer.text\_en.type" : "custom",  
> "index.analysis.analyzer.text\_en.filter.0" : "standard",  
> "index.number\_of\_shards" : "5",  
> "index.number\_of\_replicas" : "1",  
> "index.version.created" : "190899"  
> }  
> }
> 
> So I'm trying to index a small amount of documents using the BulkRequest,  
> but it's taking forever (almost 1 min). And when it finishes it fails most  
> of requests, I got only 376 docs indexed out of 1000, and a lot  
> of UnavailableShardsException[[albums][2] [2] shardIt, [0] active : Timeout  
> waiting for [1m], request:  
> org.elasticsearch.action.bulk.BulkShardRequest@db766c1].
> 
> I would imagine this is related to the shard config. I'm only running  
> using a single instance for now.
> 
> The code for the bulk load:
> 
> while(rs.next() && count \< max){
> 
> count++;
> 
> XContentBuilder content = createContent(rs);
> 
> bulkRequest.add(client.prepareIndex("albums", "album"  
> ).setSource(content));
> 
> if(count%1000 == 0){
> 
> System.out.println("commit");
> 
> BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> 
> if(bulkResponse.hasFailures()){
> 
> for(BulkItemResponse r : bulkResponse.items()){
> 
> System.out.println(r.getFailureMessage());
> 
> }
> 
> System.out.println("ERROR");
> 
> ```
> }
> 
> ```
> 
> }
> 
> }
> 
> How can I make it faster for a single node POC?
> 
> Regards

---

<div class="post-metadata">

**Author:** ![Vinicius\_Carvalho](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vinicius_carvalho/32/770_2.png) [@Vinicius\_Carvalho](https://discuss.elastic.co/u/Vinicius_Carvalho)\
**Post date:** [July 11, 2012, 9:14pm UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/4 "2012-07-11T21:14:30Z")

</div>

Well, problem was the automatic generation of id, I really would like to  
know why, I was expecting it to be safe ☹

One last thing, the process to index is now running, but again, taking a  
long, long time. Solr indexed the same data (1.7M records) in under 3 min.  
I wonder why ES is taking so long (so far, I'm sending bulk updates at  
every 10k docs read from DB) and each takes around 25sec to be indexed.

Any ideas on how to boost performance?

Regards

On Wednesday, July 11, 2012 4:55:16 PM UTC-4, Vinicius Carvalho wrote:

> Hi! fixed this by changing the shard/replica values. Still a bit slower  
> when compared to Solr DataImporthandler
> 
> Besides that, tons of errors on duplicateddocuments are arising, which  
> makes no sense, as I'm not sending duplicate documents. Only thing I could  
> think is that ids are being duplicated (I'm not sending ids, instead, I'm  
> relying on automatic generation of id)
> 
> Still not getting why.
> 
> Any ideas?
> 
> REgards
> 
> On Wednesday, July 11, 2012 3:52:53 PM UTC-4, Vinicius Carvalho wrote:
> 
> > Hi there. Well, getting my feet wet with ES. I'm doing a small poc here,  
> > and I created an index with default shard/replica values (did not touch  
> > anything). The only thing I changed was the analyzer:
> > 
> > "albums" : {  
> > "settings" : {  
> > "index.analysis.analyzer.text\_en.filter.2" : "porterStem",  
> > "index.analysis.analyzer.text\_en.filter.1" : "lowercase",  
> > "index.analysis.analyzer.text\_en.tokenizer" : "standard",  
> > "index.analysis.analyzer.text\_en.type" : "custom",  
> > "index.analysis.analyzer.text\_en.filter.0" : "standard",  
> > "index.number\_of\_shards" : "5",  
> > "index.number\_of\_replicas" : "1",  
> > "index.version.created" : "190899"  
> > }  
> > }
> > 
> > So I'm trying to index a small amount of documents using the BulkRequest,  
> > but it's taking forever (almost 1 min). And when it finishes it fails most  
> > of requests, I got only 376 docs indexed out of 1000, and a lot  
> > of UnavailableShardsException[[albums][2] [2] shardIt, [0] active : Timeout  
> > waiting for [1m], request:  
> > org.elasticsearch.action.bulk.BulkShardRequest@db766c1].
> > 
> > I would imagine this is related to the shard config. I'm only running  
> > using a single instance for now.
> > 
> > The code for the bulk load:
> > 
> > while(rs.next() && count \< max){
> > 
> > count++;
> > 
> > XContentBuilder content = createContent(rs);
> > 
> > bulkRequest.add(client.prepareIndex("albums", "album"  
> > ).setSource(content));
> > 
> > if(count%1000 == 0){
> > 
> > System.out.println("commit");
> > 
> > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > 
> > if(bulkResponse.hasFailures()){
> > 
> > for(BulkItemResponse r : bulkResponse.items()){
> > 
> > System.out.println(r.getFailureMessage());
> > 
> > }
> > 
> > System.out.println("ERROR");
> > 
> > ```
> > }
> > 
> > ```
> > 
> > }
> > 
> > }
> > 
> > How can I make it faster for a single node POC?
> > 
> > Regards

---

<div class="post-metadata">

**Author:** ![Radu\_Gheorghe1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe1/32/2688_2.png) [@Radu\_Gheorghe1](https://discuss.elastic.co/u/Radu_Gheorghe1)\
**Post date:** [July 12, 2012, 7:15am UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/5 "2012-07-12T07:15:39Z")

</div>

Hi Vinicius,

If you want to insert a huge number of docs for one time, it might be worth  
disabling the automatic refresh, insert all the docs (in that time you  
won't see them in searches), then enable the refresh back again.

Take a look here, towards the end, when it says "Bulk indexing usage":

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

On Thursday, July 12, 2012 12:14:30 AM UTC+3, Vinicius Carvalho wrote:

> Well, problem was the automatic generation of id, I really would like to  
> know why, I was expecting it to be safe ☹
> 
> One last thing, the process to index is now running, but again, taking a  
> long, long time. Solr indexed the same data (1.7M records) in under 3 min.  
> I wonder why ES is taking so long (so far, I'm sending bulk updates at  
> every 10k docs read from DB) and each takes around 25sec to be indexed.
> 
> Any ideas on how to boost performance?
> 
> Regards
> 
> On Wednesday, July 11, 2012 4:55:16 PM UTC-4, Vinicius Carvalho wrote:
> 
> > Hi! fixed this by changing the shard/replica values. Still a bit slower  
> > when compared to Solr DataImporthandler
> > 
> > Besides that, tons of errors on duplicateddocuments are arising, which  
> > makes no sense, as I'm not sending duplicate documents. Only thing I could  
> > think is that ids are being duplicated (I'm not sending ids, instead, I'm  
> > relying on automatic generation of id)
> > 
> > Still not getting why.
> > 
> > Any ideas?
> > 
> > REgards
> > 
> > On Wednesday, July 11, 2012 3:52:53 PM UTC-4, Vinicius Carvalho wrote:
> > 
> > > Hi there. Well, getting my feet wet with ES. I'm doing a small poc here,  
> > > and I created an index with default shard/replica values (did not touch  
> > > anything). The only thing I changed was the analyzer:
> > > 
> > > "albums" : {  
> > > "settings" : {  
> > > "index.analysis.analyzer.text\_en.filter.2" : "porterStem",  
> > > "index.analysis.analyzer.text\_en.filter.1" : "lowercase",  
> > > "index.analysis.analyzer.text\_en.tokenizer" : "standard",  
> > > "index.analysis.analyzer.text\_en.type" : "custom",  
> > > "index.analysis.analyzer.text\_en.filter.0" : "standard",  
> > > "index.number\_of\_shards" : "5",  
> > > "index.number\_of\_replicas" : "1",  
> > > "index.version.created" : "190899"  
> > > }  
> > > }
> > > 
> > > So I'm trying to index a small amount of documents using the  
> > > BulkRequest, but it's taking forever (almost 1 min). And when it finishes  
> > > it fails most of requests, I got only 376 docs indexed out of 1000, and a  
> > > lot of UnavailableShardsException[[albums][2] [2] shardIt, [0] active :  
> > > Timeout waiting for [1m], request:  
> > > org.elasticsearch.action.bulk.BulkShardRequest@db766c1].
> > > 
> > > I would imagine this is related to the shard config. I'm only running  
> > > using a single instance for now.
> > > 
> > > The code for the bulk load:
> > > 
> > > while(rs.next() && count \< max){
> > > 
> > > count++;
> > > 
> > > XContentBuilder content = createContent(rs);
> > > 
> > > bulkRequest.add(client.prepareIndex("albums", "album"  
> > > ).setSource(content));
> > > 
> > > if(count%1000 == 0){
> > > 
> > > System.out.println("commit");
> > > 
> > > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > > 
> > > if(bulkResponse.hasFailures()){
> > > 
> > > for(BulkItemResponse r : bulkResponse.items()){
> > > 
> > > System.out.println(r.getFailureMessage());
> > > 
> > > }
> > > 
> > > System.out.println("ERROR");
> > > 
> > > ```
> > > }
> > > 
> > > ```
> > > 
> > > }
> > > 
> > > }
> > > 
> > > How can I make it faster for a single node POC?
> > > 
> > > Regards

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [July 14, 2012, 10:50am UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/6 "2012-07-14T10:50:34Z")

</div>

From your loop code, it is not obvious to see whether you clear the  
bulkrequest after the bulkresponse had come back. If you do not clear, you  
will send already sent docs again and again.

'UnavailableShardsException' means Elasticsearch did not found a shard, but  
you addressed it in indexing.

From your configuration it seems you are on a single node with replica  
setting 1, means your cluster state is yellow. Please take care of  
this. You will be able to index on a single node without errors with  
replica setting 0 and with a cluster state of green.

Can you provide us with numbers what you compare so we can follow your  
statement "Still a bit slower when compared to Solr DataImporthandler" in  
more detail? Thanks.

Best regards,

Jörg

On Wednesday, July 11, 2012 10:55:16 PM UTC+2, Vinicius Carvalho wrote:

> Hi! fixed this by changing the shard/replica values. Still a bit slower  
> when compared to Solr DataImporthandler
> 
> Besides that, tons of errors on duplicateddocuments are arising, which  
> makes no sense, as I'm not sending duplicate documents. Only thing I could  
> think is that ids are being duplicated (I'm not sending ids, instead, I'm  
> relying on automatic generation of id)
> 
> Still not getting why.
> 
> Any ideas?
> 
> REgards
> 
> On Wednesday, July 11, 2012 3:52:53 PM UTC-4, Vinicius Carvalho wrote:
> 
> > Hi there. Well, getting my feet wet with ES. I'm doing a small poc here,  
> > and I created an index with default shard/replica values (did not touch  
> > anything). The only thing I changed was the analyzer:
> > 
> > "albums" : {  
> > "settings" : {  
> > "index.analysis.analyzer.text\_en.filter.2" : "porterStem",  
> > "index.analysis.analyzer.text\_en.filter.1" : "lowercase",  
> > "index.analysis.analyzer.text\_en.tokenizer" : "standard",  
> > "index.analysis.analyzer.text\_en.type" : "custom",  
> > "index.analysis.analyzer.text\_en.filter.0" : "standard",  
> > "index.number\_of\_shards" : "5",  
> > "index.number\_of\_replicas" : "1",  
> > "index.version.created" : "190899"  
> > }  
> > }
> > 
> > So I'm trying to index a small amount of documents using the BulkRequest,  
> > but it's taking forever (almost 1 min). And when it finishes it fails most  
> > of requests, I got only 376 docs indexed out of 1000, and a lot  
> > of UnavailableShardsException[[albums][2] [2] shardIt, [0] active : Timeout  
> > waiting for [1m], request:  
> > org.elasticsearch.action.bulk.BulkShardRequest@db766c1].
> > 
> > I would imagine this is related to the shard config. I'm only running  
> > using a single instance for now.
> > 
> > The code for the bulk load:
> > 
> > while(rs.next() && count \< max){
> > 
> > count++;
> > 
> > XContentBuilder content = createContent(rs);
> > 
> > bulkRequest.add(client.prepareIndex("albums", "album"  
> > ).setSource(content));
> > 
> > if(count%1000 == 0){
> > 
> > System.out.println("commit");
> > 
> > BulkResponse bulkResponse = bulkRequest.execute().actionGet();
> > 
> > if(bulkResponse.hasFailures()){
> > 
> > for(BulkItemResponse r : bulkResponse.items()){
> > 
> > System.out.println(r.getFailureMessage());
> > 
> > }
> > 
> > System.out.println("ERROR");
> > 
> > ```
> > }
> > 
> > ```
> > 
> > }
> > 
> > }
> > 
> > How can I make it faster for a single node POC?
> > 
> > Regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:20am UTC](https://discuss.elastic.co/t/bulk-insertion-taking-long-and-throwing-lots-of-errors/8390/7 "2017-07-06T03:20:19Z")

</div>