Hi, we are running
Elasticsearch 7.6.2 on our own in AWS.
- 6 data nodes on
- 3 dedicated master nodes
- 200 indices storing ~800million documents.
- Total size ~5TB
- 30s refresh interval
We are observing high CPU usage most of the time (above 60-80% with spikes at 100%).
Although the performance of the cluster seems acceptable:
- ingesting ~500 docs/second
- no rejections on bulk thread pool
- query times look good
I'm a bit concerned about the CPU usage, because we expect more data from our customers, and we are going to add more nodes or even increase the specs of the existing ones to handle all the additional load.
This is the output from
It seems like Elasticsearch is spending a lot of time merging.
I would like to understand if such a high CPU usage is expected on a well-balanced cluster, especially when it's mostly bulk indexing.
I tried to increase
refresh_interval but the CPU usage didn't change at all. GC activity changed a bit for sure, and the merge sizes increased but the CPU remained high.
Thanking in advance.