# Elasticsearch num processors/bulk threads detection **URL:** \ **Category:** Elasticsearch\ **Created:** [July 2, 2016, 5:39pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584 "2016-07-02T17:39:46Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![abhijith\_reddy](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@abhijith\_reddy](https://discuss.elastic.co/u/abhijith_reddy)\ **Post date:** [July 2, 2016, 5:39pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/1 "2016-07-02T17:39:46Z")
How does elasticsearch detect the number of available processors ? Each node in our cluster has 48 cores but the bulk thread pool size is 32, just trying to find out how elasticsearch is arriving at the number. Below is the output from `curl -XGET localhost:9200/_nodes/` for one of the nodes. ``` "cpu" : { "vendor" : "Intel", "model" : "Xeon", "mhz" : 2501, "total_cores" : 48, "total_sockets" : 1, "cores_per_socket" : 32, "cache_size_in_bytes" : 30720 }, ``` ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 3, 2016, 2:46pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/2 "2016-07-03T14:46:00Z")
What you see is an information from the sigar library, which is known to return wrong values [https://github.com/hyperic/sigar/issues/63](https://github.com/hyperic/sigar/issues/63) but these values were never used for ES internal configuration. Sigar has been removed since May 2015 > so I wonder what Elasticsearch version you use. ES uses `Runtime.getRuntime().availableProcessors()` which depends on the JVM implementation. Use Java 8+ and Elasticsearch 2+ for best results. ---
**Author:** ![abhijith\_reddy](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@abhijith\_reddy](https://discuss.elastic.co/u/abhijith_reddy)\ **Post date:** [July 3, 2016, 4:26pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/3 "2016-07-03T16:26:47Z")
Thanks ! Thats interesting, but I am seeing the same behavior on 1.4 and 2.3.3 (with java 8u20) as well. Also when I create a simple java program and run it on the machine the output of `Runtime.getRuntime().availableProcessors()` is 48. ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [July 3, 2016, 6:25pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/4 "2016-07-03T18:25:41Z")
I looked into the code, Elasticsearch has a maximum limit of 32 for processor-based thread pools. See `org.elasticsearch.common.util.concurrent.EsExecutors` ```auto /** * Returns the number of processors available but at most 32. */ public static int boundedNumberOfProcessors(Settings settings) { /* This relates to issues where machines with large number of cores * ie. >= 48 create too many threads and run into OOM see #3478 * We just use an 32 core upper-bound here to not stress the system * too much with too many created threads */ int defaultValue = Math.min(32, Runtime.getRuntime().availableProcessors()); try { defaultValue = Integer.parseInt(System.getProperty(DEFAULT_SYSPROP)); } catch (Throwable ignored) {} return settings.getAsInt(PROCESSORS, defaultValue); } ``` The reason for the limit is explained at > While I share the diagnosis, I do not follow why a local limit of 32 can ensure to prevent "thread explosions", since Elasticsearch has six thread pools which are proportional to the available processor count: INDEX, BULK, GET, SEARCH, SUGGEST, PERCOLATE. Plus, there are five thread pools that are 50% of the processor count, but not more than 5 or 10 threads: LISTENER, FLUSH, REFREH, WARMER, SNAPSHOT. And finally, there are two pools with double size of the processor count: FETCH\_SHARD\_STARTED and FETCH\_SHARD\_STORE. That can result in 6_32 + 3_5 + 2_10 + 2_2\*32 = 192 + 15 + 20 + 128 = 335 threads in the ES pools on 32+ core machines. Plus, there are threads running by Netty. Anyway, we can conclude there are still enough ES/Netty threads in most situations that can be scheduled with maximum efficiency on any existing CPU hardware by the operating system. Users with machines of \>32 cores and a very, very specific or exotic workload pattern might want a deeper insight to the question if exceeding the pool size limit can also increase efficiency. The power of a single action type execution on a node is bound by a hard-coded maximum of 32 threads, but I doubt if that boundary is really a problem or can be measured at all. My machines have a core count of 36 and 40 but maybe if I find time it is possible to run benchmarks with mixed search/index workload, enabling/disabling the hard-coded limit. ---
**Author:** ![abhijith\_reddy](https://avatars.discourse-cdn.com/v4/letter/a/977dab/32.png) [@abhijith\_reddy](https://discuss.elastic.co/u/abhijith_reddy)\ **Post date:** [July 3, 2016, 7:38pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/5 "2016-07-03T19:38:05Z")
Thanks a lot ! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:38pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/6 "2017-07-05T22:38:47Z")