Hi All,
I am setting high number of thread by threadpool.bulk.size: 160 in elasticsearch.yml
But when I am getting maximum 4 threads.
I am using 4 core CPU.
What can be the issue ?
Thanks!
What version are you using? You can't no longer set threadpool.bulk.size more then the number of processors/cores:
It doesn't make any sense to set the number of threads that high. If they are all scheduable then they'll just starve each other of CPU time, and you'll get worse performance than if you had set it lower (from all the context switching). As @msimos pointed out, starting in 2.1.2 and 2.2.0 of Elasticsearch, it will not be possible to set the number of threads in the bulk and indexing pools to more than the number of cores; Elasticsearch will warn you that this is bad and truncate the value to the number of cores. Starting in Elasticsearch 5.0.0, we will relax this slightly to allow one plus the number of cores (so another thread is available if one is blocked on I/O) but Elasticsearch will hard fail at startup if you try to set it higher.
Cool.
That's useful information.
Thanks!
One follow up query.
If we add more machine will that add to bulk thread capacity. So suppose I add one more machine with 4 processor now total two nodes in cluster. Will my bulk thread become 8 or still remain 4 ?
Thread pool settings apply per Java VM. No matter how many nodes you run, you don't have to adjust the thread pool settings. As a side note, you should run one Java VM per machine.
Hi @jprante
I am not running multiple jvm on single machine.
I am asking if I add one more node to server ( separate server/machine) . So it will be 2 separate servers and separate JVM.
In this case will the bulk thread capacity doubled ?