Bulk Queue Size and Thread Count

[dawiro]

Hi,
Is there any relationship between bulk queue size and bulk thread count? Does it increase in proportion with the number of threads?

Regards,
D

[dbakti7 (Dian Bakti)]

Bulk thread count by default is equivalent to number of available processors.
The default queue size is 200.
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/modules-threadpool.html
(Note: in more recent version of ES, "bulk" thread pool has been renamed to "write" thread pool, in case you are reading newer documentation).

I'm not sure what you mean with "increase in proportion with the number of threads"
If you are referring to worker threads, then yes, the more concurrent threads that you run, the more likely the number of tasks that get queued increase (because your bulk processors cannot catch up).
Once your queue size exceeds 200, the task might start getting rejected.
For example: if you have 8 cores, your bulk thread pool size will be 8
if you are firing 10 concurrent bulk request: 8 of them will be actively worked on, the other 2 will be in queue and only get executed once one of the thread finished.