Thanks Alex & Jorg.
can you explain what kind of problems you have?
The problem is getting EsRejectedExecutionException during bulk indexing.
This error started appearing after our move to 0.90.9.
We wanted to save on memory so have set bloom filter settings as false
initially. In fact we migrated to 0.90.9 for this change only.
And why you think that not loading bloom filters will solve it?
Thinking was with bloom filter false the indexing time will increase and
that may fill up the queue fast so tried with true as well.
But the error is coming independent of this setting.
yes checking with
http://localhost:9200/_nodes/thread_pool/stats?pretty=true for knowing the
bulk rejections.
the soundest method is to check your bulk indexing code for capping the
number of concurrent bulk requests. Or you may increase the number of
nodes/shards. > Right now it seems your concurrency setting is too high for
the capacity of your cluster (just a wild guess, without knowing your
numbers)
out setup:
4 shards per index. max 150 indices or so & have single node setup.
In our case 1 bulk call has 2000 entries to be indexed & 4 concurrent bulk
calls.
Will also reduce bulk entries size and max 3 concurrent bulk calls and
check on this again to see if that helps. Thanks.
Btw since current indexing rate was working fine with 0.90.1 this got
introduced with 0.90.9 for sure against our bulk indexing rate.
Thanks,
Ram
On Monday, February 3, 2014 4:35:49 PM UTC+5:30, Jörg Prante wrote:
The new setting of bounded bulk queue is a precaution not to overwhelm the
internal ES buffers.
Unbounded queue size is potentially dangerous since your node operations
may get stalled by too many bulk requests.
You could increase bulk queue_size from 50 to 100 if that works better.
But, the soundest method is to check your bulk indexing code for capping
the number of concurrent bulk requests. Or you may increase the number of
nodes/shards. Right now it seems your concurrency setting is too high for
the capacity of your cluster (just a wild guess, without knowing your
numbers).
Jörg
On Fri, Jan 31, 2014 at 6:48 AM, Ramchandra Phadake <
ramchandr...@gmail.com <javascript:>> wrote:
Hi,
Recently we migrated from 0.90.1 to 0.90.9. We did change
index.codec.bloom.load=false initially but even with true we obsetve same
behaviour.
Nothing else has changed from indexing code. We are using bulk API. Any
pointers why we can get EsRejectedExecutionException?
This doesn't frequently in indexing process.
We did try by setting with few thousand size but this exception comes but
rarely.
After setting bulk queue to -1 it appears to be working. -1 means
unbounded size so is it safe to set to -1.
Thanks,
Ram
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/89c6e465-c23f-4234-9667-cd09be4a7178%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17fabc2a-7cda-4e15-8ec3-eb6662ca5ca8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.