Elasticsearch with marvel-agent frequently crashed!


(Symol Lin) #1

  We have a experimental elasticsearch cluster. However, the master node of them frequently crashed! There is the log.

[2016-02-11 06:40:09,188][INFO ][monitor.jvm              ] [node-1] [gc][young][53015][3954] duration [708ms], collections [1]/[1.4s], total [708ms]/[46.7s], memory [171mb]->[173.7mb]/[1015.6mb], all_pools {[young] [24.1mb]->[31.7mb]/[66.5mb]}{[survivor] [2.7mb]->[3.6mb]/[8.3mb]}{[old] [144.1mb]->[144.1mb]/[940.8mb]}
[2016-02-11 06:52:55,906][WARN ][monitor.jvm              ] [node-1] [gc][young][53118][3966] duration [7.8s], collections [2]/[1m], total [7.8s]/[56.8s], memory [197.1mb]->[197mb]/[1015.6mb], all_pools {[young] [48.9mb]->[46.5mb]/[66.5mb]}{[survivor] [5mb]->[993.4kb]/[8.3mb]}{[old] [145mb]->[149.8mb]/[940.8mb]}
[2016-02-11 07:54:37,060][ERROR][marvel.agent.collector.indices] [node-1] collector [{}] timed out when collecting data
[2016-02-11 08:23:44,857][ERROR][marvel.agent.collector.cluster] [node-1] collector [{}] timed out when collecting data
[2016-02-11 09:14:50,675][WARN ][monitor.jvm              ] [node-1] [gc][young][53121][3967] duration [3.2s], collections [1]/[2.3h], total [3.2s]/[1m], memory [203.8mb]->[187.9mb]/[1015.6mb], all_pools {[young] [53.6mb]->[36.6mb]/[66.5mb]}{[survivor] [993.4kb]->[1.7mb]/[8.3mb]}{[old] [149.8mb]->[149.8mb]/[940.8mb]}

  There is no more useful message. Could anyone tell me what happened to my cluster?


(system) #2