Hi, I started using elasticsearch 5.0.1
I want to increase my index or bulk thread pool size in settings but i get the error
Failed to parse value [..] for setting [thread_pool.index.size] must be <= ....;
In documentation (https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-threadpool.html) I see that:
For index/delete operations. Thread pool type is fixed with a size of # of available processors, queue_size of 200. The maximum size for this pool is 1 + # of available processors.
But the problem is, elasticsearch is heavily queueing my bulk operations. In previous versions(ex:2.1.1) I could increase this limit and the problem was being solved, indexing was pretty fast and no queue overloading. But even if elasticsearch is using its own implementation of queue, it does not seem to work better with limiting thread size. Now I cannot increase limit and use fast indexing because elasticsearch forces threadpool to a limit.
Is there a way to increase this limit? In previous versions I could increase the limit and os was managing its own thread queue better then elasticsearch. This may depend my data size, network, cpu performance etc. but, I should be able to specify this limit nonetheless.
It seems like the only way to do is to override number of processors that elasticsearch is using. If there a better way?