ElasticSearch high CPU on merge threads


I am doing a bulk indexing every 10 minutes using logstash to send data to ElasticSearch. For almost 20 days everything went OK, but today I am getting the following issue:

  • ElasticSearch is using very high CPU , and when I am getting hot threads it seems that almost all of the big cpu thead are the Lucene merging threads.

  • My bulk indexing is lasting 4 times longer.

I am running 2 elasticsearch nodes on SSD disk.

Does anyone knows where the problem could come from ?

What version are you on?
Are there lots of deleted documents or segments?

I run ES version 2.2.1 and I dont delete docs at all

Are you updating a lot of documents?

Neither, basically I bulk index around 30000 docs in a new index every 10
minutes with no update/delete. My master node seems to be totally slower
than before. My search became slower as well. I really can t tell if it is
an ES or an hardware problem as I did a full cluster restart

Do you have Marvel installed to monitor your cluster? If not that'd be a good idea as it'll give you something to look at in terms of metrics.

I will install marvel and check this out. Thanks

Could not install Marvel but the problem is solved. The problem was with the server itself and not ElasticSearch. Could not detect exactly where it was from but a full server restart solved it.