We recently upgraded from ES version 5.6.10 to 7.3.1 and noticed indexing performance seems much worse. We could index at a rate of around 500k op/s, and now we peak at around 200k op/s.
During indexing, we get bulk request thread rejections, but we are actually only utilising around 50% of our available CPU, and it seems we are not able to increase the thread pool size because of this limit here: https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/threadpool/ExecutorBuilder.java#L52
Our use case is that we only reindex clusters that don't receive live traffic, so I don't think this limit really makes sense for us. I wondered what the reasoning behind it was, and if other people have the same issue? I assume if we could increase the thread pool size, we would be able to utilise ~100% of CPU for indexing.
Steps to reproduce :
- Reindex a cluster with enough writes per second to get bulk thread rejections
- Check if CPU is actually maxing out
- In our case, we're only using 50% CPU