How to find the thread pool size of an ElasticSearch cluster?

truong_ha · October 20, 2014, 10:09am

I'm writing the concurrent code to send bulk index to ElasticSearch, and
sending this query to get the thread pool size:

GET /_cat/thread_pool?v&h=host,bulk.active,bulk.queueSize

The response is

host bulk.active bulk.queueSize
1D4HPY1 0 50
1D4HPY2 0 50
1D4HPY3 0 50
1D4HPY4 0 50

So how can I calculate the actual pool size of that cluster? Is it the sum
of all hosts which means 200?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · October 20, 2014, 11:09am

bulk.queueSize is the maximum size before requests are rejected.

Jörg

On Mon, Oct 20, 2014 at 12:09 PM, truong ha truonghatsts@gmail.com wrote:

I'm writing the concurrent code to send bulk index to Elasticsearch, and
sending this query to get the thread pool size:

GET /_cat/thread_pool?v&h=host,bulk.active,bulk.queueSize

The response is

host bulk.active bulk.queueSize
1D4HPY1 0 50
1D4HPY2 0 50
1D4HPY3 0 50
1D4HPY4 0 50

So how can I calculate the actual pool size of that cluster? Is it the sum
of all hosts which means 200?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHBLvxUvLweqDQUQOs2bP6OxJPs3TgJkFLKWk%2BZi6Bgkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

truong_ha · October 20, 2014, 1:57pm

So in my case, which is the maximum requests I can send: 200 or 50?

On Monday, 20 October 2014 18:09:28 UTC+7, Jörg Prante wrote:

bulk.queueSize is the maximum size before requests are rejected.

Jörg

On Mon, Oct 20, 2014 at 12:09 PM, truong ha <truong...@gmail.com
<javascript:>> wrote:

I'm writing the concurrent code to send bulk index to Elasticsearch, and
sending this query to get the thread pool size:

GET /_cat/thread_pool?v&h=host,bulk.active,bulk.queueSize

The response is

host bulk.active bulk.queueSize
1D4HPY1 0 50
1D4HPY2 0 50
1D4HPY3 0 50
1D4HPY4 0 50

So how can I calculate the actual pool size of that cluster? Is it the
sum of all hosts which means 200?

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/29373a56-dbe7-4afb-9733-78f3fbbf5c2d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

jprante · October 20, 2014, 3:25pm

This is not the maximum number of requests you can send. It means "when
bulk indexing on a node gets too busy and must be queued, the maximum
number of actions that are allowed to queue up before a client is notified
of rejections is 50".

Jörg

On Mon, Oct 20, 2014 at 3:57 PM, truong ha truonghatsts@gmail.com wrote:

So in my case, which is the maximum requests I can send: 200 or 50?

On Monday, 20 October 2014 18:09:28 UTC+7, Jörg Prante wrote:

bulk.queueSize is the maximum size before requests are rejected.

Jörg

On Mon, Oct 20, 2014 at 12:09 PM, truong ha truong...@gmail.com wrote:

I'm writing the concurrent code to send bulk index to Elasticsearch, and
sending this query to get the thread pool size:

GET /_cat/thread_pool?v&h=host,bulk.active,bulk.queueSize

The response is

host bulk.active bulk.queueSize
1D4HPY1 0 50
1D4HPY2 0 50
1D4HPY3 0 50
1D4HPY4 0 50

So how can I calculate the actual pool size of that cluster? Is it the
sum of all hosts which means 200?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/29373a56-dbe7-4afb-9733-78f3fbbf5c2d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/29373a56-dbe7-4afb-9733-78f3fbbf5c2d%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF%3D0RY90k%2BS7-Y-RNhcjshdSptWZUoPQx7GC5r0AMXJkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

truong_ha · October 21, 2014, 6:33am

50 is the maximum for one node, so 200 is the maximum for the entire
cluster right?

It means to avoid being rejected, I should gently send requests to the
cluster, not to overwhelm the queue?

On Monday, 20 October 2014 22:27:26 UTC+7, Jörg Prante wrote:

This is not the maximum number of requests you can send. It means "when
bulk indexing on a node gets too busy and must be queued, the maximum
number of actions that are allowed to queue up before a client is notified
of rejections is 50".

Jörg

On Mon, Oct 20, 2014 at 3:57 PM, truong ha <truong...@gmail.com
<javascript:>> wrote:

So in my case, which is the maximum requests I can send: 200 or 50?

On Monday, 20 October 2014 18:09:28 UTC+7, Jörg Prante wrote:

bulk.queueSize is the maximum size before requests are rejected.

Jörg

On Mon, Oct 20, 2014 at 12:09 PM, truong ha truong...@gmail.com wrote:

I'm writing the concurrent code to send bulk index to Elasticsearch,
and sending this query to get the thread pool size:

GET /_cat/thread_pool?v&h=host,bulk.active,bulk.queueSize

The response is

host bulk.active bulk.queueSize
1D4HPY1 0 50
1D4HPY2 0 50
1D4HPY3 0 50
1D4HPY4 0 50

So how can I calculate the actual pool size of that cluster? Is it the
sum of all hosts which means 200?

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e6e48fe3-6269-493c-9258-fdc97baeb27e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/29373a56-dbe7-4afb-9733-78f3fbbf5c2d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/29373a56-dbe7-4afb-9733-78f3fbbf5c2d%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/66161b58-eeca-4277-9e81-12f323a8181a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.