High CPU consumption


(Eugene Strokin) #1

Hello,
I'm experiencing very high CPU usage, nearly 100%. The documentation says that ES is light on CPU. I assume that it could be some heavy queries. But how can I prove it and find what queries are eating CPU?
I've ran hot_threads:

But I don't understand what could be the problem here?
Please help,
Thanks,
Eugene


(Mark Walkom) #2

Looks like you may have an overloaded cluster, can you add another node?


(Eugene Strokin) #3

Thanks, looks like it is the case, I've added another node, CPU usage dropped.
Out of curiosity, did you get to this conclusion based on the hot_threads info? If so, what exactly shows the overload?
Thanks,
Eugene


(Harlin) #4

Are any of the queries coming in sorted queries or aggregations?


(Eugene Strokin) #5

We have a lot of sorted queries. This is the business needs. No aggregations though, as far as I can see.


(Harlin) #6

Sorted queries can cause issues with Elasticsearch if the cluster isn't prepared for them. (getting individual fields to sort on out of an inverted index is hard). ES has better ways of getting this field data. If you aren't already using one of these approaches I would high suggest either eagerly loading field data or using doc values


(Haochun) #7

@Eugene_Strokin i have encountered the same probole,did you find the cause?i want to find which query,but i have no way:sob:


(Eugene Strokin) #8

After careful investigation I noticed that a lot of sorted queries are running, and I assumed the Harlin's suggestion is correct. There is no way for me to change the queries, so I had to add shards and replicas to the number which pushed down the CPU usage to acceptable level, but still considerably high.


(system) #9