Node stats question - threads and active

Elastic Stack Elasticsearch
1

Hi,
I recently changed thread_pool type of "bulk" from the "cached" (default) to "blocking".
This was because I observed very high number of threads (about 20,000) and often times it ended up with losing the node. (by slow response of the node).
I thought the number of thread itself was choking the system, so I'm trying the blocking type.
Since changing it to the blocking, the system is stable. (so far. I changed it only a couple of days ago.)

By the way, the question here is about node stats.
The curl localhost:9200/_cluster/nodes//stats?thread_pool=true gives us
"threads", "queue", "active" for each thread pool.
I understand "queue" is the number of tasks waiting in the thread pool queue.
What I don't understand is "threads" and "active".
My assumption is "thread" is the number of threads created in the thread pool and "active" is the number of thread which is currently running.
Is this correct?
It seems "threads" is simply growing. Does it decrease some time later in some condition?

Are there difference by the type of the thread pool?
I can imagine "queue" has nothing to do with "cached", "threads" do not change with "fixed".
So the meaningful measure for "cached" is "threads" (or "active"?) and for "fixed" is "queue" and "active"?

I'm trying to monitor the node's 'busyness' by monitoring the thread_pool in node stats.

Thanks for your help.

2

Your assumptions about threads and active are correct. The threads is the
number of current threads in the pool and active is approximate number of
threads that are actively executing tasks. The number of threads in the
pool should drop down to the minimum number of threads (specified by min
parameter and defaults to 1) after a period of inactivity (specified by
keep_alive parameter and defaults to 5min). You can find more information
about different types of threads
here: Elasticsearch Platform — Find real-time answers at scale | Elastic

On Monday, November 12, 2012 7:42:38 PM UTC-5, arta wrote:

Hi,
I recently changed thread_pool type of "bulk" from the "cached" (default)
to
"blocking".
This was because I observed very high number of threads (about 20,000) and
often times it ended up with losing the node. (by slow response of the
node).
I thought the number of thread itself was choking the system, so I'm
trying
the blocking type.
Since changing it to the blocking, the system is stable. (so far. I
changed
it only a couple of days ago.)

By the way, the question here is about node stats.
The curl localhost:9200/_cluster/nodes//stats?thread_pool=true
gives us
"threads", "queue", "active" for each thread pool.
I understand "queue" is the number of tasks waiting in the thread pool
queue.
What I don't understand is "threads" and "active".
My assumption is "thread" is the number of threads created in the thread
pool and "active" is the number of thread which is currently running.
Is this correct?
It seems "threads" is simply growing. Does it decrease some time later in
some condition?

Are there difference by the type of the thread pool?
I can imagine "queue" has nothing to do with "cached", "threads" do not
change with "fixed".
So the meaningful measure for "cached" is "threads" (or "active"?) and for
"fixed" is "queue" and "active"?

I'm trying to monitor the node's 'busyness' by monitoring the thread_pool
in
node stats.

Thanks for your help.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Node-stats-question-threads-and-active-tp4025348.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--

3

Thank you Igor,
I'll be monitoring "active" for both "cached" and "blocking" thread pools, then.
(I don't have "fixed" thread pool).
Thanks again.