# Rejected execution (queue capacity 50) in bulk process **URL:** \ **Category:** Elasticsearch\ **Created:** [March 4, 2014, 3:45pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149 "2014-03-04T15:45:09Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 4, 2014, 3:45pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/1 "2014-03-04T15:45:09Z")
Hello, I'm facing a problem bulk indexing 5k documents in 24 different indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. I've read that a thread per index is used, that would mean I'm using 24 bulk threads at one time. Am I right? if so, why I'm getting this rejection since queue capacity is 50? It's possible that replicas consume threads as well? or a thread per shard is used? this is my \_cluster/health: { "cluster\_name": "elasticsearch", "status": "yellow", "timed\_out": false, "number\_of\_nodes": 1, "number\_of\_data\_nodes": 1, "active\_primary\_shards": 120, "active\_shards": 120, "relocating\_shards": 0, "initializing\_shards": 0, "unassigned\_shards": 120 } 120 shards = 24 indices \* 5 shard I've tried splitting the bulk indexing in different chunk sizes with same result (as far as i understand this is not the solution). Am I doing something wrong or I just have to increase the queue size? I will appreciate any help thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 4, 2014, 4:55pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/2 "2014-03-04T16:55:40Z")
Threads are pooled, they are not used per index. The queue length of 50 works in almost any case. 50 is also safe to protect a node before being overwhelmed by too many documents. If not, think about the bulk request size, and if your cluster is powerful enough for processing the transmitted documents. You can add 1000-10000 requests in a single bulk request. If you still exhaust the bulk queue size, you should examine your code if you really examine the bulk responses of each bulk request, and if your bulk client limits the concurrency of bulk requests. Look at the BulkProcessor source code to learn about concurrent bulk requests. Jörg On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > Hello, > > I'm facing a problem bulk indexing 5k documents in 24 different indices > (i18n). I'm using elasticsearch 1.0.1 with all default settings. I've read > that a thread per index is used, that would mean I'm using 24 bulk threads > at one time. Am I right? if so, why I'm getting this rejection since queue > capacity is 50? It's possible that replicas consume threads as well? or a > thread per shard is used? this is my \_cluster/health: > > { > "cluster\_name": "elasticsearch", > "status": "yellow", > "timed\_out": false, > "number\_of\_nodes": 1, > "number\_of\_data\_nodes": 1, > "active\_primary\_shards": 120, > "active\_shards": 120, > "relocating\_shards": 0, > "initializing\_shards": 0, > "unassigned\_shards": 120 > } > > 120 shards = 24 indices \* 5 shard > > I've tried splitting the bulk indexing in different chunk sizes with same > result (as far as i understand this is not the solution). > > Am I doing something wrong or I just have to increase the queue size? > > I will appreciate any help > > thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 4, 2014, 5:12pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/3 "2014-03-04T17:12:49Z")
I understand, but I must be missing something. 5k documents \* 24 indices = 120k requests, How am i supposed to bulk index them? I've tried to set "index.refresh\_interval" to "-1" to speed up the process but still same result. splitting the bulk in different sizes didn't work either. On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Threads are pooled, they are not used per index. > > The queue length of 50 works in almost any case. 50 is also safe to > protect a node before being overwhelmed by too many documents. If not, > think about the bulk request size, and if your cluster is powerful enough > for processing the transmitted documents. You can add 1000-10000 requests > in a single bulk request. > > If you still exhaust the bulk queue size, you should examine your code if > you really examine the bulk responses of each bulk request, and if your > bulk client limits the concurrency of bulk requests. Look at the > BulkProcessor source code to learn about concurrent bulk requests. > > Jörg > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > Hello, > > > > I'm facing a problem bulk indexing 5k documents in 24 different indices > > (i18n). I'm using elasticsearch 1.0.1 with all default settings. I've read > > that a thread per index is used, that would mean I'm using 24 bulk threads > > at one time. Am I right? if so, why I'm getting this rejection since queue > > capacity is 50? It's possible that replicas consume threads as well? or a > > thread per shard is used? this is my \_cluster/health: > > > > { > > "cluster\_name": "elasticsearch", > > "status": "yellow", > > "timed\_out": false, > > "number\_of\_nodes": 1, > > "number\_of\_data\_nodes": 1, > > "active\_primary\_shards": 120, > > "active\_shards": 120, > > "relocating\_shards": 0, > > "initializing\_shards": 0, > > "unassigned\_shards": 120 > > } > > > > 120 shards = 24 indices \* 5 shard > > > > I've tried splitting the bulk indexing in different chunk sizes with same > > result (as far as i understand this is not the solution). > > > > Am I doing something wrong or I just have to increase the queue size? > > > > I will appreciate any help > > > > thanks > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 4, 2014, 5:43pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/4 "2014-03-04T17:43:51Z")
Without being able to look at source code, it is difficult if not impossible to find issues. "index.refresh\_interval: -1" must be set to the respective index, preferably using the cluster update API (conf file or index creation settings is also possible but not a good place for temporary settings). It does not "speed up" bulk indexing, it saves some resources, the node does not refresh that often for index reads, which is often a heavy operation. With only 120k docs, distributed over 24 indices, the effect may not be visible at all. Default flush buffer size for an index is 64k IIRC. Jörg On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > I understand, but I must be missing something. > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > the process but still same result. splitting the bulk in different sizes > didn't work either. > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > Threads are pooled, they are not used per index. > > > > The queue length of 50 works in almost any case. 50 is also safe to > > protect a node before being overwhelmed by too many documents. If not, > > think about the bulk request size, and if your cluster is powerful enough > > for processing the transmitted documents. You can add 1000-10000 requests > > in a single bulk request. > > > > If you still exhaust the bulk queue size, you should examine your code if > > you really examine the bulk responses of each bulk request, and if your > > bulk client limits the concurrency of bulk requests. Look at the > > BulkProcessor source code to learn about concurrent bulk requests. > > > > Jörg > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > > > Hello, > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different indices > > > (i18n). I'm using elasticsearch 1.0.1 with all default settings. I've read > > > that a thread per index is used, that would mean I'm using 24 bulk threads > > > at one time. Am I right? if so, why I'm getting this rejection since queue > > > capacity is 50? It's possible that replicas consume threads as well? or a > > > thread per shard is used? this is my \_cluster/health: > > > > > > { > > > "cluster\_name": "elasticsearch", > > > "status": "yellow", > > > "timed\_out": false, > > > "number\_of\_nodes": 1, > > > "number\_of\_data\_nodes": 1, > > > "active\_primary\_shards": 120, > > > "active\_shards": 120, > > > "relocating\_shards": 0, > > > "initializing\_shards": 0, > > > "unassigned\_shards": 120 > > > } > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > I've tried splitting the bulk indexing in different chunk sizes with > > > same result (as far as i understand this is not the solution). > > > > > > Am I doing something wrong or I just have to increase the queue size? > > > > > > I will appreciate any help > > > > > > thanks > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > . > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 4, 2014, 5:59pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/5 "2014-03-04T17:59:04Z")
This is what I'm doing: for each 24 locales: POST /current\_contenidos\_[LOCALE] # creates index PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = -1 POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 indices]) for each 24 locales: PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Without being able to look at source code, it is difficult if not > impossible to find issues. > > "index.refresh\_interval: -1" must be set to the respective index, > preferably using the cluster update API (conf file or index creation > settings is also possible but not a good place for temporary settings). It > does not "speed up" bulk indexing, it saves some resources, the node does > not refresh that often for index reads, which is often a heavy operation. > With only 120k docs, distributed over 24 indices, the effect may not be > visible at all. Default flush buffer size for an index is 64k IIRC. > > Jörg > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > I understand, but I must be missing something. > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > > the process but still same result. splitting the bulk in different sizes > > didn't work either. > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > Threads are pooled, they are not used per index. > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > protect a node before being overwhelmed by too many documents. If not, > > > think about the bulk request size, and if your cluster is powerful enough > > > for processing the transmitted documents. You can add 1000-10000 requests > > > in a single bulk request. > > > > > > If you still exhaust the bulk queue size, you should examine your code > > > if you really examine the bulk responses of each bulk request, and if your > > > bulk client limits the concurrency of bulk requests. Look at the > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > Jörg > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > > > > > Hello, > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different indices > > > > (i18n). I'm using elasticsearch 1.0.1 with all default settings. I've read > > > > that a thread per index is used, that would mean I'm using 24 bulk threads > > > > at one time. Am I right? if so, why I'm getting this rejection since queue > > > > capacity is 50? It's possible that replicas consume threads as well? or a > > > > thread per shard is used? this is my \_cluster/health: > > > > > > > > { > > > > "cluster\_name": "elasticsearch", > > > > "status": "yellow", > > > > "timed\_out": false, > > > > "number\_of\_nodes": 1, > > > > "number\_of\_data\_nodes": 1, > > > > "active\_primary\_shards": 120, > > > > "active\_shards": 120, > > > > "relocating\_shards": 0, > > > > "initializing\_shards": 0, > > > > "unassigned\_shards": 120 > > > > } > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes with > > > > same result (as far as i understand this is not the solution). > > > > > > > > Am I doing something wrong or I just have to increase the queue size? > > > > > > > > I will appreciate any help > > > > > > > > thanks > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 4, 2014, 6:13pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/6 "2014-03-04T18:13:27Z")
So you use plain HTTP API? Do you evaluate the responses from POST /\_bulk requests before sending next request? Jörg On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > This is what I'm doing: > > for each 24 locales: > POST /current\_contenidos\_[LOCALE] # creates index > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = -1 > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > indices]) > > for each 24 locales: > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > Without being able to look at source code, it is difficult if not > > impossible to find issues. > > > > "index.refresh\_interval: -1" must be set to the respective index, > > preferably using the cluster update API (conf file or index creation > > settings is also possible but not a good place for temporary settings). It > > does not "speed up" bulk indexing, it saves some resources, the node does > > not refresh that often for index reads, which is often a heavy operation. > > With only 120k docs, distributed over 24 indices, the effect may not be > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > Jörg > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > I understand, but I must be missing something. > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > > > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > > > the process but still same result. splitting the bulk in different sizes > > > didn't work either. > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > > protect a node before being overwhelmed by too many documents. If not, > > > > think about the bulk request size, and if your cluster is powerful enough > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > in a single bulk request. > > > > > > > > If you still exhaust the bulk queue size, you should examine your code > > > > if you really examine the bulk responses of each bulk request, and if your > > > > bulk client limits the concurrency of bulk requests. Look at the > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > Jörg > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > > > > > > > Hello, > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > { > > > > > "cluster\_name": "elasticsearch", > > > > > "status": "yellow", > > > > > "timed\_out": false, > > > > > "number\_of\_nodes": 1, > > > > > "number\_of\_data\_nodes": 1, > > > > > "active\_primary\_shards": 120, > > > > > "active\_shards": 120, > > > > > "relocating\_shards": 0, > > > > > "initializing\_shards": 0, > > > > > "unassigned\_shards": 120 > > > > > } > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes with > > > > > same result (as far as i understand this is not the solution). > > > > > > > > > > Am I doing something wrong or I just have to increase the queue size? > > > > > > > > > > I will appreciate any help > > > > > > > > > > thanks > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > . > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 4, 2014, 6:19pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/7 "2014-03-04T18:19:01Z")
I'm using python, I just sent the log so you could figure out what I'm doing. I'm not evaluating the bulk response but logging them so I can see the 'rejected execution' error in most of them. On 4 March 2014 19:13, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > So you use plain HTTP API? Do you evaluate the responses from POST /\_bulk > requests before sending next request? > > Jörg > > On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > This is what I'm doing: > > > > for each 24 locales: > > POST /current\_contenidos\_[LOCALE] # creates index > > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = -1 > > > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > > indices]) > > > > for each 24 locales: > > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > Without being able to look at source code, it is difficult if not > > > impossible to find issues. > > > > > > "index.refresh\_interval: -1" must be set to the respective index, > > > preferably using the cluster update API (conf file or index creation > > > settings is also possible but not a good place for temporary settings). It > > > does not "speed up" bulk indexing, it saves some resources, the node does > > > not refresh that often for index reads, which is often a heavy operation. > > > With only 120k docs, distributed over 24 indices, the effect may not be > > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > > > Jörg > > > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > I understand, but I must be missing something. > > > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > > > > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > > > > the process but still same result. splitting the bulk in different sizes > > > > didn't work either. > > > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > > > protect a node before being overwhelmed by too many documents. If not, > > > > > think about the bulk request size, and if your cluster is powerful enough > > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > > in a single bulk request. > > > > > > > > > > If you still exhaust the bulk queue size, you should examine your code > > > > > if you really examine the bulk responses of each bulk request, and if your > > > > > bulk client limits the concurrency of bulk requests. Look at the > > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > > > Jörg > > > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > > > { > > > > > > "cluster\_name": "elasticsearch", > > > > > > "status": "yellow", > > > > > > "timed\_out": false, > > > > > > "number\_of\_nodes": 1, > > > > > > "number\_of\_data\_nodes": 1, > > > > > > "active\_primary\_shards": 120, > > > > > > "active\_shards": 120, > > > > > > "relocating\_shards": 0, > > > > > > "initializing\_shards": 0, > > > > > > "unassigned\_shards": 120 > > > > > > } > > > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes with > > > > > > same result (as far as i understand this is not the solution). > > > > > > > > > > > > Am I doing something wrong or I just have to increase the queue size? > > > > > > > > > > > > I will appreciate any help > > > > > > > > > > > > thanks > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0\_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 4, 2014, 6:53pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/8 "2014-03-04T18:53:12Z")
Logging is not enough, you should care for the number of active requests sent and the bulk request responses that came back. So you can control the number of concurrent bulk requests that are active at a time, and if you do so, you can limit this number, before exceeding the bulk queue size of 50. Jörg On Tue, Mar 4, 2014 at 7:19 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > I'm using python, I just sent the log so you could figure out what I'm > doing. > > I'm not evaluating the bulk response but logging them so I can see the > 'rejected execution' error in most of them. > > On 4 March 2014 19:13, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > So you use plain HTTP API? Do you evaluate the responses from POST /\_bulk > > requests before sending next request? > > > > Jörg > > > > On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > This is what I'm doing: > > > > > > for each 24 locales: > > > POST /current\_contenidos\_[LOCALE] # creates index > > > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = -1 > > > > > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > > > indices]) > > > > > > for each 24 locales: > > > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > > > > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > Without being able to look at source code, it is difficult if not > > > > impossible to find issues. > > > > > > > > "index.refresh\_interval: -1" must be set to the respective index, > > > > preferably using the cluster update API (conf file or index creation > > > > settings is also possible but not a good place for temporary settings). It > > > > does not "speed up" bulk indexing, it saves some resources, the node does > > > > not refresh that often for index reads, which is often a heavy operation. > > > > With only 120k docs, distributed over 24 indices, the effect may not be > > > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > > > > > Jörg > > > > > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > I understand, but I must be missing something. > > > > > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > > > > > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > > > > > the process but still same result. splitting the bulk in different sizes > > > > > didn't work either. > > > > > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > > > > protect a node before being overwhelmed by too many documents. If not, > > > > > > think about the bulk request size, and if your cluster is powerful enough > > > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > > > in a single bulk request. > > > > > > > > > > > > If you still exhaust the bulk queue size, you should examine your > > > > > > code if you really examine the bulk responses of each bulk request, and if > > > > > > your bulk client limits the concurrency of bulk requests. Look at the > > > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > > > > > Jörg > > > > > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > > > > > { > > > > > > > "cluster\_name": "elasticsearch", > > > > > > > "status": "yellow", > > > > > > > "timed\_out": false, > > > > > > > "number\_of\_nodes": 1, > > > > > > > "number\_of\_data\_nodes": 1, > > > > > > > "active\_primary\_shards": 120, > > > > > > > "active\_shards": 120, > > > > > > > "relocating\_shards": 0, > > > > > > > "initializing\_shards": 0, > > > > > > > "unassigned\_shards": 120 > > > > > > > } > > > > > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes with > > > > > > > same result (as far as i understand this is not the solution). > > > > > > > > > > > > > > Am I doing something wrong or I just have to increase the queue size? > > > > > > > > > > > > > > I will appreciate any help > > > > > > > > > > > > > > thanks > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0\_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 4, 2014, 7:02pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/9 "2014-03-04T19:02:26Z")
Ok, I'm gonna play with that, but it seems too complicated just for bulk indexing taking into account the low number of documents Thanks El 04/03/2014 19:53, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) escribió: > Logging is not enough, you should care for the number of active requests > sent and the bulk request responses that came back. So you can control the > number of concurrent bulk requests that are active at a time, and if you do > so, you can limit this number, before exceeding the bulk queue size of 50. > > Jörg > > On Tue, Mar 4, 2014 at 7:19 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > I'm using python, I just sent the log so you could figure out what I'm > > doing. > > > > I'm not evaluating the bulk response but logging them so I can see the > > 'rejected execution' error in most of them. > > > > On 4 March 2014 19:13, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > So you use plain HTTP API? Do you evaluate the responses from POST > > > /\_bulk requests before sending next request? > > > > > > Jörg > > > > > > On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > This is what I'm doing: > > > > > > > > for each 24 locales: > > > > POST /current\_contenidos\_[LOCALE] # creates index > > > > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = -1 > > > > > > > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > > > > indices]) > > > > > > > > for each 24 locales: > > > > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > > > > > > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > Without being able to look at source code, it is difficult if not > > > > > impossible to find issues. > > > > > > > > > > "index.refresh\_interval: -1" must be set to the respective index, > > > > > preferably using the cluster update API (conf file or index creation > > > > > settings is also possible but not a good place for temporary settings). It > > > > > does not "speed up" bulk indexing, it saves some resources, the node does > > > > > not refresh that often for index reads, which is often a heavy operation. > > > > > With only 120k docs, distributed over 24 indices, the effect may not be > > > > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > > > > > > > Jörg > > > > > > > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > I understand, but I must be missing something. > > > > > > > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to bulk > > > > > > index them? I've tried to set "index.refresh\_interval" to "-1" to speed up > > > > > > the process but still same result. splitting the bulk in different sizes > > > > > > didn't work either. > > > > > > > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > > > > > protect a node before being overwhelmed by too many documents. If not, > > > > > > > think about the bulk request size, and if your cluster is powerful enough > > > > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > > > > in a single bulk request. > > > > > > > > > > > > > > If you still exhaust the bulk queue size, you should examine your > > > > > > > code if you really examine the bulk responses of each bulk request, and if > > > > > > > your bulk client limits the concurrency of bulk requests. Look at the > > > > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > > > > > > > { > > > > > > > > "cluster\_name": "elasticsearch", > > > > > > > > "status": "yellow", > > > > > > > > "timed\_out": false, > > > > > > > > "number\_of\_nodes": 1, > > > > > > > > "number\_of\_data\_nodes": 1, > > > > > > > > "active\_primary\_shards": 120, > > > > > > > > "active\_shards": 120, > > > > > > > > "relocating\_shards": 0, > > > > > > > > "initializing\_shards": 0, > > > > > > > > "unassigned\_shards": 120 > > > > > > > > } > > > > > > > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes > > > > > > > > with same result (as far as i understand this is not the solution). > > > > > > > > > > > > > > > > Am I doing something wrong or I just have to increase the queue > > > > > > > > size? > > > > > > > > > > > > > > > > I will appreciate any help > > > > > > > > > > > > > > > > thanks > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > To unsubscribe from this topic, visit > > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > > . > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > > > > . > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0\_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 4, 2014, 8:33pm UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/10 "2014-03-04T20:33:37Z")
Probably you should check why your cluster is yellow - is that a single node only? Bulk indexing with a green cluster should work flawlessly. Jörg On Tue, Mar 4, 2014 at 8:02 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > Ok, I'm gonna play with that, but it seems too complicated just for bulk > indexing taking into account the low number of documents > > Thanks > El 04/03/2014 19:53, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > escribió: > > > Logging is not enough, you should care for the number of active requests > > sent and the bulk request responses that came back. So you can control the > > number of concurrent bulk requests that are active at a time, and if you do > > so, you can limit this number, before exceeding the bulk queue size of 50. > > > > Jörg > > > > On Tue, Mar 4, 2014 at 7:19 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > I'm using python, I just sent the log so you could figure out what I'm > > > doing. > > > > > > I'm not evaluating the bulk response but logging them so I can see the > > > 'rejected execution' error in most of them. > > > > > > On 4 March 2014 19:13, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > So you use plain HTTP API? Do you evaluate the responses from POST > > > > /\_bulk requests before sending next request? > > > > > > > > Jörg > > > > > > > > On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > This is what I'm doing: > > > > > > > > > > for each 24 locales: > > > > > POST /current\_contenidos\_[LOCALE] # creates index > > > > > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = > > > > > -1 > > > > > > > > > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > > > > > indices]) > > > > > > > > > > for each 24 locales: > > > > > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > > > > > > > > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > > > Without being able to look at source code, it is difficult if not > > > > > > impossible to find issues. > > > > > > > > > > > > "index.refresh\_interval: -1" must be set to the respective index, > > > > > > preferably using the cluster update API (conf file or index creation > > > > > > settings is also possible but not a good place for temporary settings). It > > > > > > does not "speed up" bulk indexing, it saves some resources, the node does > > > > > > not refresh that often for index reads, which is often a heavy operation. > > > > > > With only 120k docs, distributed over 24 indices, the effect may not be > > > > > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > > > > > > > > > Jörg > > > > > > > > > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > I understand, but I must be missing something. > > > > > > > > > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to > > > > > > > bulk index them? I've tried to set "index.refresh\_interval" to "-1" to > > > > > > > speed up the process but still same result. splitting the bulk in different > > > > > > > sizes didn't work either. > > > > > > > > > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe to > > > > > > > > protect a node before being overwhelmed by too many documents. If not, > > > > > > > > think about the bulk request size, and if your cluster is powerful enough > > > > > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > > > > > in a single bulk request. > > > > > > > > > > > > > > > > If you still exhaust the bulk queue size, you should examine your > > > > > > > > code if you really examine the bulk responses of each bulk request, and if > > > > > > > > your bulk client limits the concurrency of bulk requests. Look at the > > > > > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > > > > > > > > > { > > > > > > > > > "cluster\_name": "elasticsearch", > > > > > > > > > "status": "yellow", > > > > > > > > > "timed\_out": false, > > > > > > > > > "number\_of\_nodes": 1, > > > > > > > > > "number\_of\_data\_nodes": 1, > > > > > > > > > "active\_primary\_shards": 120, > > > > > > > > > "active\_shards": 120, > > > > > > > > > "relocating\_shards": 0, > > > > > > > > > "initializing\_shards": 0, > > > > > > > > > "unassigned\_shards": 120 > > > > > > > > > } > > > > > > > > > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes > > > > > > > > > with same result (as far as i understand this is not the solution). > > > > > > > > > > > > > > > > > > Am I doing something wrong or I just have to increase the queue > > > > > > > > > size? > > > > > > > > > > > > > > > > > > I will appreciate any help > > > > > > > > > > > > > > > > > > thanks > > > > > > > > > > > > > > > > > > -- > > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > > Groups "elasticsearch" group. > > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > > > > > > > To view this discussion on the web visit > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > > > > > . > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this topic, visit > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > > > . > > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > > > > > . > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > > > > > . > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0\_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa\_kSnpnBrOZAgJC\_zcA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa_kSnpnBrOZAgJC_zcA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jose\_Gargallo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jose_gargallo/32/1752_2.png) [@Jose\_Gargallo](https://discuss.elastic.co/u/Jose_Gargallo)\ **Post date:** [March 5, 2014, 11:58am UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/11 "2014-03-05T11:58:01Z")
Going green worked for 1 shard and 0 replicas, but dind't work for 5 shards and 0 replicas being green. Thanks On 4 March 2014 21:33, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > Probably you should check why your cluster is yellow - is that a single > node only? > > Bulk indexing with a green cluster should work flawlessly. > > Jörg > > On Tue, Mar 4, 2014 at 8:02 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com) wrote: > > > Ok, I'm gonna play with that, but it seems too complicated just for bulk > > indexing taking into account the low number of documents > > > > Thanks > > El 04/03/2014 19:53, "[joergprante@gmail.com](mailto:joergprante@gmail.com)" [joergprante@gmail.com](mailto:joergprante@gmail.com) > > escribió: > > > > > Logging is not enough, you should care for the number of active requests > > > sent and the bulk request responses that came back. So you can control the > > > number of concurrent bulk requests that are active at a time, and if you do > > > so, you can limit this number, before exceeding the bulk queue size of 50. > > > > > > Jörg > > > > > > On Tue, Mar 4, 2014 at 7:19 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > I'm using python, I just sent the log so you could figure out what I'm > > > > doing. > > > > > > > > I'm not evaluating the bulk response but logging them so I can see the > > > > 'rejected execution' error in most of them. > > > > > > > > On 4 March 2014 19:13, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > So you use plain HTTP API? Do you evaluate the responses from POST > > > > > /\_bulk requests before sending next request? > > > > > > > > > > Jörg > > > > > > > > > > On Tue, Mar 4, 2014 at 6:59 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > This is what I'm doing: > > > > > > > > > > > > for each 24 locales: > > > > > > POST /current\_contenidos\_[LOCALE] # creates index > > > > > > PUT /current\_contenidos\_es\_es/\_settings # sets refresh\_interval = > > > > > > -1 > > > > > > > > > > > > POST /\_bulk HTTP/1.1" 200 2386284 (12 times [mixed requests for 24 > > > > > > indices]) > > > > > > > > > > > > for each 24 locales: > > > > > > PUT /contenidos\_es\_es/\_settings # sets back refresh\_interval = 1s > > > > > > > > > > > > On 4 March 2014 18:43, [joergprante@gmail.com](mailto:joergprante@gmail.com) [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > > > > > > > > > > > Without being able to look at source code, it is difficult if not > > > > > > > impossible to find issues. > > > > > > > > > > > > > > "index.refresh\_interval: -1" must be set to the respective index, > > > > > > > preferably using the cluster update API (conf file or index creation > > > > > > > settings is also possible but not a good place for temporary settings). It > > > > > > > does not "speed up" bulk indexing, it saves some resources, the node does > > > > > > > not refresh that often for index reads, which is often a heavy operation. > > > > > > > With only 120k docs, distributed over 24 indices, the effect may not be > > > > > > > visible at all. Default flush buffer size for an index is 64k IIRC. > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > On Tue, Mar 4, 2014 at 6:12 PM, Jose Gargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > > > I understand, but I must be missing something. > > > > > > > > > > > > > > > > 5k documents \* 24 indices = 120k requests, How am i supposed to > > > > > > > > bulk index them? I've tried to set "index.refresh\_interval" to "-1" to > > > > > > > > speed up the process but still same result. splitting the bulk in different > > > > > > > > sizes didn't work either. > > > > > > > > > > > > > > > > On 4 March 2014 17:55, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com) > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Threads are pooled, they are not used per index. > > > > > > > > > > > > > > > > > > The queue length of 50 works in almost any case. 50 is also safe > > > > > > > > > to protect a node before being overwhelmed by too many documents. If not, > > > > > > > > > think about the bulk request size, and if your cluster is powerful enough > > > > > > > > > for processing the transmitted documents. You can add 1000-10000 requests > > > > > > > > > in a single bulk request. > > > > > > > > > > > > > > > > > > If you still exhaust the bulk queue size, you should examine your > > > > > > > > > code if you really examine the bulk responses of each bulk request, and if > > > > > > > > > your bulk client limits the concurrency of bulk requests. Look at the > > > > > > > > > BulkProcessor source code to learn about concurrent bulk requests. > > > > > > > > > > > > > > > > > > Jörg > > > > > > > > > > > > > > > > > > On Tue, Mar 4, 2014 at 4:45 PM, jgargallo [jgargallo@gmail.com](mailto:jgargallo@gmail.com)wrote: > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > I'm facing a problem bulk indexing 5k documents in 24 different > > > > > > > > > > indices (i18n). I'm using elasticsearch 1.0.1 with all default settings. > > > > > > > > > > I've read that a thread per index is used, that would mean I'm using 24 > > > > > > > > > > bulk threads at one time. Am I right? if so, why I'm getting this rejection > > > > > > > > > > since queue capacity is 50? It's possible that replicas consume threads as > > > > > > > > > > well? or a thread per shard is used? this is my \_cluster/health: > > > > > > > > > > > > > > > > > > > > { > > > > > > > > > > "cluster\_name": "elasticsearch", > > > > > > > > > > "status": "yellow", > > > > > > > > > > "timed\_out": false, > > > > > > > > > > "number\_of\_nodes": 1, > > > > > > > > > > "number\_of\_data\_nodes": 1, > > > > > > > > > > "active\_primary\_shards": 120, > > > > > > > > > > "active\_shards": 120, > > > > > > > > > > "relocating\_shards": 0, > > > > > > > > > > "initializing\_shards": 0, > > > > > > > > > > "unassigned\_shards": 120 > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > 120 shards = 24 indices \* 5 shard > > > > > > > > > > > > > > > > > > > > I've tried splitting the bulk indexing in different chunk sizes > > > > > > > > > > with same result (as far as i understand this is not the solution). > > > > > > > > > > > > > > > > > > > > Am I doing something wrong or I just have to increase the queue > > > > > > > > > > size? > > > > > > > > > > > > > > > > > > > > I will appreciate any help > > > > > > > > > > > > > > > > > > > > thanks > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > You received this message because you are subscribed to the > > > > > > > > > > Google Groups "elasticsearch" group. > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > > > > > > > > > > > > To view this discussion on the web visit > > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/88614f79-e1df-420a-b471-fb0eedb9baa9%40googlegroups.com) > > > > > > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > > > > > -- > > > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > > > To unsubscribe from this topic, visit > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > > > > . > > > > > > > > > To unsubscribe from this group and all its topics, send an email > > > > > > > > > to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > > To view this discussion on the web visit > > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY\_Oj4CtrA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH2aRoc58Luz1Y4o-v73q1-v6UwV0NWnEq8YY_Oj4CtrA%40mail.gmail.com) > > > > > > > > > . > > > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOW1UAk7SiLfO%3D912dSi56ahYazzNjEPGMAKyeR%2BLkzwQ%40mail.gmail.com) > > > > > > > > . > > > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > To unsubscribe from this topic, visit > > > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > > > . > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEdv4usqhoSe32w-sqz3ZBSn2LwKTH2BqanUU4t70nE%3Dw%40mail.gmail.com) > > > > > > > . > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B\_THGMZbbQygtVjNXv1zA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOMyPrV5Uf0KZ98PVARGQFdc%2B_THGMZbbQygtVjNXv1zA%40mail.gmail.com) > > > > > > . > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK\_pCQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFvXLcJp6%2Bz3RnLmTwQQa%3DX1Msaigcyw9E2zs8JBK_pCQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0\_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaOm5UfqeYdRA54ObktgJ0_ZDRGmDiaT8uZmi311LqTRoA%40mail.gmail.com) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq\_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-\_hFQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGNPf6-7Ckxq_6RDfnRZ0WTT20yP%3DK0qdC6U%2B62m-_hFQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaNqVtXx%3DFuw2-OmUmgM3MGCK2fjtcvoMhBmVcBEB46gxw%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe](https://groups.google.com/d/topic/elasticsearch/AemPEOkDvEE/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa\_kSnpnBrOZAgJC\_zcA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa_kSnpnBrOZAgJC_zcA%40mail.gmail.com)[https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa\_kSnpnBrOZAgJC\_zcA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFS%3D9J6DuBfF53QG3Ai420qoFZa_kSnpnBrOZAgJC_zcA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPKnLaO9WmyBrDdaUC-2rPmYqXWeLR5CQ%3DH7gULfh%3DHOgr1uEQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPKnLaO9WmyBrDdaUC-2rPmYqXWeLR5CQ%3DH7gULfh%3DHOgr1uEQ%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:45am UTC](https://discuss.elastic.co/t/rejected-execution-queue-capacity-50-in-bulk-process/16149/12 "2017-07-06T01:45:36Z")