ESRejectedExecutionException

Elastic Stack Elasticsearch
1

Hi,

  I have 5 million records in database i create the index through

BulkRequestBuilder every 10,000 records pushing to elasticsearch But while
sometimes getting ESRejectedExecutionException

ES_MIN=2gb
ES_MAX=2gb

cluster.name: pandy
node.name: "pandy_server"
index.number_of_shards: 5
index.number_of_replicas: 1
bootstrap.mlockall: true
index.store.compress.stored: true
index.refresh_interval: 1m
indices.memory.index_buffer_size: 90%
threadpool.search.type: fixed
threadpool.search.size: 15
index.cache.filter.expire: 1m
indices.cache.filter.size: 256mb
threadpool.index.type: fixed
threadpool.index.size: 25
threadpool.bulk.type: fixed
threadpool.bulk.size: 60

 I have *only 6GB* RAM

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/47b41f84-6dcc-407b-aca1-6fdf4ba4ce1a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

The bulk thread pool has a cap on the queue size which is 50
(threadpool.bulk.queue_size). Once you reach that cap, you will start
getting the rejected exceptions. That is there for a reason so that the
thread pool does not get out of control. You can try to increase the queue
size slightly but if that still doesn't fix your problem, then you need to
reduce your bulk concurrency/size since you may be reaching the limits of
what your node hardware can handle.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e23e9e51-4793-4eb8-91a7-8604a2f83b6b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Binh,

    Thanks for your reply.I am setting for *threadpool.bulk.queue_size:

300* in elasticsearch.yml it's okay.

On Thursday, March 27, 2014 6:00:42 PM UTC+5:30, Binh Ly wrote:

The bulk thread pool has a cap on the queue size which is 50
(threadpool.bulk.queue_size). Once you reach that cap, you will start
getting the rejected exceptions. That is there for a reason so that the
thread pool does not get out of control. You can try to increase the queue
size slightly but if that still doesn't fix your problem, then you need to
reduce your bulk concurrency/size since you may be reaching the limits of
what your node hardware can handle.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/af661c4b-8750-4733-9f53-eaf86f866bd5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

On Thursday, March 27, 2014 11:23:52 AM UTC+5:30, Pandiyan Arumugam wrote:

Hi,

  I have 5 million records in database i create the index through

BulkRequestBuilder every 10,000 records pushing to elasticsearch But while
sometimes getting ESRejectedExecutionException

ES_MIN=2gb
ES_MAX=2gb

cluster.name: pandy
node.name: "pandy_server"
index.number_of_shards: 5
index.number_of_replicas: 1
bootstrap.mlockall: true
index.store.compress.stored: true
index.refresh_interval: 1m
indices.memory.index_buffer_size: 90%
threadpool.search.type: fixed
threadpool.search.size: 15
index.cache.filter.expire: 1m
indices.cache.filter.size: 256mb
threadpool.index.type: fixed
threadpool.index.size: 25
threadpool.bulk.type: fixed
threadpool.bulk.size: 60

 I have *only 6GB* RAM

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/86d8b037-07d2-4986-b0ac-b3c607457716%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.