Queue Size limited?

Elastic Stack Elasticsearch
1

Hi all,

last restart of our cluster caused a heavy load as a couple of rivers are
triggered which load alltogether about 40 GB of data into the (single-node)
cluster.

This lead to massive exceptions like shown in the logs:
[6726]: index [my_index], type [my_doc_type], id [2390342], message
[EsRejectedExecutionException[rejected execution (queue capacity 50) on
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@4a94944]]

We didn't change any of the threadpools queue size settings (like
threadpool.bulk.queue_size). AFAIK in this case the queue size is unlimited.

So how is it possible that the exception above can occur? Why does the log
message indicate a queue size of 50?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7f095b58-e471-43a7-9d20-d35dabd1a697%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Is this with JDBC river? If so, you can increase the number of bulk actions
per request, to decrease the concurrent bulk actions, to remedy situations
where many rivers are active.

Jörg

On Mon, Feb 23, 2015 at 9:40 AM, Abid Hussain hussain@novacom.mygbiz.com
wrote:

Hi all,

last restart of our cluster caused a heavy load as a couple of rivers are
triggered which load alltogether about 40 GB of data into the (single-node)
cluster.

This lead to massive exceptions like shown in the logs:
[6726]: index [my_index], type [my_doc_type], id [2390342], message
[EsRejectedExecutionException[rejected execution (queue capacity 50) on
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@4a94944
]]

We didn't change any of the threadpools queue size settings (like
threadpool.bulk.queue_size). AFAIK in this case the queue size is
unlimited.

So how is it possible that the exception above can occur? Why does the log
message indicate a queue size of 50?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7f095b58-e471-43a7-9d20-d35dabd1a697%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7f095b58-e471-43a7-9d20-d35dabd1a697%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE6wXT%2BstLnyadVxXvi0-pzycMn9Nfbjv7AR7eABjaw7Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Yes it happened with jdbc river. Thanks for clarification.

Am Montag, 23. Februar 2015 17:16:09 UTC+1 schrieb Jörg Prante:

Is this with JDBC river? If so, you can increase the number of bulk
actions per request, to decrease the concurrent bulk actions, to remedy
situations where many rivers are active.

Jörg

On Mon, Feb 23, 2015 at 9:40 AM, Abid Hussain <hus...@novacom.mygbiz.com
<javascript:>> wrote:

Hi all,

last restart of our cluster caused a heavy load as a couple of rivers are
triggered which load alltogether about 40 GB of data into the (single-node)
cluster.

This lead to massive exceptions like shown in the logs:
[6726]: index [my_index], type [my_doc_type], id [2390342], message
[EsRejectedExecutionException[rejected execution (queue capacity 50) on
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1@4a94944]]

We didn't change any of the threadpools queue size settings (like
threadpool.bulk.queue_size). AFAIK in this case the queue size is
unlimited.

So how is it possible that the exception above can occur? Why does the
log message indicate a queue size of 50?

Regards,

Abid

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7f095b58-e471-43a7-9d20-d35dabd1a697%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/7f095b58-e471-43a7-9d20-d35dabd1a697%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b7edddf8-a619-4a3d-a25f-8f8a40af3e36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.