# Elasticsearch num processors/bulk threads detection

**URL:** <https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584>\
**Category:** Elasticsearch\
**Created:** [July 2, 2016, 5:39pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584 "2016-07-02T17:39:46Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [July 3, 2016, 6:25pm UTC](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584/4 "2016-07-03T18:25:41Z")

</div>

I looked into the code, Elasticsearch has a maximum limit of 32 for processor-based thread pools.

See `org.elasticsearch.common.util.concurrent.EsExecutors`

```auto
    /**
     * Returns the number of processors available but at most <tt>32</tt>.
     */
    public static int boundedNumberOfProcessors(Settings settings) {
        /* This relates to issues where machines with large number of cores
         * ie. >= 48 create too many threads and run into OOM see #3478
         * We just use an 32 core upper-bound here to not stress the system
         * too much with too many created threads */
        int defaultValue = Math.min(32, Runtime.getRuntime().availableProcessors());
        try {
            defaultValue = Integer.parseInt(System.getProperty(DEFAULT_SYSPROP));
        } catch (Throwable ignored) {}
        return settings.getAsInt(PROCESSORS, defaultValue);
    }

```

The reason for the limit is explained at

> <https://github.com/elastic/elasticsearch/issues/3545>

While I share the diagnosis, I do not follow why a local limit of 32 can ensure to prevent "thread explosions", since Elasticsearch has six thread pools which are proportional to the available processor count: INDEX, BULK, GET, SEARCH, SUGGEST, PERCOLATE. Plus, there are five thread pools that are 50% of the processor count, but not more than 5 or 10 threads: LISTENER, FLUSH, REFREH, WARMER, SNAPSHOT. And finally, there are two pools with double size of the processor count: FETCH\_SHARD\_STARTED and FETCH\_SHARD\_STORE. That can result in 6_32 + 3_5 + 2_10 + 2_2\*32 = 192 + 15 + 20 + 128 = 335 threads in the ES pools on 32+ core machines. Plus, there are threads running by Netty. Anyway, we can conclude there are still enough ES/Netty threads in most situations that can be scheduled with maximum efficiency on any existing CPU hardware by the operating system.

Users with machines of \>32 cores and a very, very specific or exotic workload pattern might want a deeper insight to the question if exceeding the pool size limit can also increase efficiency. The power of a single action type execution on a node is bound by a hard-coded maximum of 32 threads, but I doubt if that boundary is really a problem or can be measured at all.

My machines have a core count of 36 and 40 but maybe if I find time it is possible to run benchmarks with mixed search/index workload, enabling/disabling the hard-coded limit.

---

_[View the full topic](https://discuss.elastic.co/t/elasticsearch-num-processors-bulk-threads-detection/54584)._