GC Problem

Mark_Poole · March 27, 2014, 4:52pm

Hey guys.

I had a strange problem with a 1.1.0: I have a 3 node cluster and
yesterday one of the nodes started to run GC one after another, causing the
cluster to become unavailable until I restarted the one node.

gist.github.com

https://gist.github.com/reedox/9812169

gistfile1.txt


Time	Event
3/26/14 
6:13:07.217 PM	
[2014-03-26 18:13:07,217][WARN ][monitor.jvm              ] [node.company.net] [gc][old][740388][8643] duration [13.1s], collections [1]/[13.2s], total [13.1s]/[1.1h], memory [11.7gb]->[11.7gb]/[11.7gb], all_pools {[young] [260.9mb]->[265.8mb]/[266.2mb]}{[survivor] [0b]->[0b]/[33.2mb]}{[old] [11.4gb]->[11.4gb]/[11.4gb]}
host = node.company.net source = /var/log/elasticsearch/companyapp.log sourcetype = elasticsearch_companyapp_log
3/26/14 
6:12:53.922 PM	
[2014-03-26 18:12:53,922][INFO ][monitor.jvm              ] [node.company.net] [gc][old][740387][8642] duration [9.9s], collections [1]/[10s], total [9.9s]/[1.1h], memory [11.7gb]->[11.7gb]/[11.7gb], all_pools {[young] [266.2mb]->[260.9mb]/[266.2mb]}{[survivor] [3.5mb]->[0b]/[33.2mb]}{[old] [11.4gb]->[11.4gb]/[11.4gb]}
host = node.company.net source = /var/log/elasticsearch/companyapp.log sourcetype = elasticsearch_companyapp_log

This file has been truncated. show original

Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f5d5b636-ee3e-4cba-9c45-b419d2b0bd21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mark_Poole · March 27, 2014, 5:10pm

Apologies for the follow up, i'm running 1.0.1

On Thursday, March 27, 2014 9:52:48 AM UTC-7, Mark Poole wrote:

Hey guys.

I had a strange problem with a 1.1.0: I have a 3 node cluster and
yesterday one of the nodes started to run GC one after another, causing the
cluster to become unavailable until I restarted the one node.

gist:9812169 · GitHub

Any ideas?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2e9c233a-530b-4655-bde2-78adb6c397ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Binh_Ly_2 · March 27, 2014, 6:36pm

I'm assuming this log is on a single node. Looking at it, it seems like the
GC just couldn't free anything up so is it possible that you just
physically need more RAM than 11G on this node to handle the load on it?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/faa2dfa1-0586-40a7-b7e3-a68ea1a35ddb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Topic		Replies	Views
Frequently gc in one node Elasticsearch	6	1226	May 28, 2018
Help with GC configuration Elasticsearch	5	652	July 6, 2017
Elasticsearch GC timeout on data node Elasticsearch	2	393	August 10, 2021
Debug Old GC after startup and on indexing (40 second hangs every other minute) Elasticsearch	1	635	July 5, 2017
Elasticsearch High CPU Usage - GC Not Working Elasticsearch	26	7054	July 5, 2017

GC Problem

Related topics