we have 8 clustered nodes and each nodes have 1 replica.
total document size is about 4GB and 1,984,173 docs.
I was suffering from very high CPU usage 80%~90% every hourly.
It is held for 5 min.
there is no other process except es on each server.
there is no other cron job even at that time.
I thought there are something wrong with es process.
maybe external attacks or gc problem.. I don't know.
It happened every hourly.
I don't know what's going on elasticsearch at this time!!
somebody help me, tell me what happened in there. please..
$ ./elasticsearch -v
Version: 1.4.2, Build: 927caff/2014-12-16T14:11:12Z, JVM: 1.7.0_75
$ java -version
java version "1.7.0_75"
Java(TM) SE Runtime Environment (build 1.7.0_75-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode)
and I installed plugins - HQ, bigdesk, head, kopf, sense
heres bigdesk graphs at cpu peak time:
<nabble_img src="es_cpu_high.png" border="0"/>
From what I can see in your graphs I noticed two things. You seem to have
a spike in search requests at that time, a spike in http traffic, and a
cache eviction right at the beginning of it.
Are you certain you don't have an external user with a cron job that runs
at the top of the hour? Perhaps a large scan and scroll query that dumps
alot of data?
Take a look at your network graphs to see if you have a correlated spike in
traffic to your Elasticsearch cluster. I wouldn't expect with a cluster
that size that you don't have any users, probably quite a few users. It
would not be unreasonable to expect that one such user is doing something
beyond what you had intended and causing stress on your system.
On Monday, March 30, 2015 at 8:37:54 PM UTC-6, vincent Park wrote:
we have 8 clustered nodes and each nodes have 1 replica.
total document size is about 4GB and 1,984,173 docs.
I was suffering from very high CPU usage 80%~90% every hourly.
It is held for 5 min.
there is no other process except es on each server.
there is no other cron job even at that time.
I thought there are something wrong with es process.
maybe external attacks or gc problem.. I don't know.
It happened every hourly.
I don't know what's going on elasticsearch at this time!!
somebody help me, tell me what happened in there. please..
$ ./elasticsearch -v
Version: 1.4.2, Build: 927caff/2014-12-16T14:11:12Z, JVM: 1.7.0_75
$ java -version
java version "1.7.0_75"
Java(TM) SE Runtime Environment (build 1.7.0_75-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode)
and I installed plugins - HQ, bigdesk, head, kopf, sense
heres bigdesk graphs at cpu peak time:
<
http://elasticsearch-users.115913.n3.nabble.com/file/n4072788/es_cpu_high.png>
--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/elasticsearch-high-cpu-usage-every-hourly-tp4072788.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aca13c50-3c3a-4630-a941-4116dc31e55e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
