Frequently young gc: is memory leak in elastic 2.3.0?

Elastic Stack Elasticsearch
1

the jvm parameters is :

Pasted image1901×77 13.3 KB

the text is:
/usr/java/jdk1.7.0_79/bin/java -Xms32g -Xmx32g -Xmn12g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/data/soft/elasticsearch_goods -cp /data/soft/elasticsearch_goods/lib/elasticsearch-2.3.0.jar:/data/soft/elasticsearch_goods/lib/* org.elasticsearch.bootstrap.Elasticsearch start -d --security.manager.enabled false

and found young gc in the logs:

Pasted image1911×458 88.1 KB

the text show:
156802 [2016-08-18 15:18:21,528][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492643][39653] duration [753ms], collections [1]/[1.1s], total [753ms]/[6.3h], memory [14.9gb]->[5.7gb]/[30.8gb], all_pools {[young] [9.3gb]->[57.4mb]/[9.6gb]}{[survivor] [110.3mb]->[253.4mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156803 [2016-08-18 15:18:30,855][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492652][39654] duration [734ms], collections [1]/[1.3s], total [734ms]/[6.3h], memory [14.8gb]->[5.7gb]/[30.8gb], all_pools {[young] [9.1gb]->[48.9mb]/[9.6gb]}{[survivor] [253.4mb]->[229.3mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156804 [2016-08-18 15:18:40,271][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492661][39655] duration [737ms], collections [1]/[1.4s], total [737ms]/[6.3h], memory [14.6gb]->[5.6gb]/[30.8gb], all_pools {[young] [9gb]->[48.4mb]/[9.6gb]}{[survivor] [229.3mb]->[144.4mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156805 [2016-08-18 15:18:49,966][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492670][39656] duration [735ms], collections [1]/[1.6s], total [735ms]/[6.3h], memory [13.9gb]->[5.6gb]/[30.8gb], all_pools {[young] [8.4gb]->[55.7mb]/[9.6gb]}{[survivor] [144.4mb]->[133.4mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156806 [2016-08-18 15:19:01,989][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492682][39657] duration [766ms], collections [1]/[1s], total [766ms]/[6.3h], memory [14.7gb]->[5.7gb]/[30.8gb], all_pools {[young] [9.1gb]->[132.8mb]/[9.6gb]}{[survivor] [133.4mb]->[187.7mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156807 [2016-08-18 15:19:11,579][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492691][39658] duration [730ms], collections [1]/[1.5s], total [730ms]/[6.3h], memory [14.9gb]->[5.6gb]/[30.8gb], all_pools {[young] [9.3gb]->[40.6mb]/[9.6gb]}{[survivor] [187.7mb]->[148.7mb]/[1.1gb]}{[old] [5.4gb]->[5.4gb]/[20gb]}
156973 [2016-08-18 15:19:21,784][INFO ][monitor.jvm ] [es_192.168.16.240] [gc][young][2492701][39659] duration [733ms], collections [1]/[1.1s], total [733ms]/[6.3h], memory [1

is there anybody encounter this problem?

2

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them..

3

thanks, I have update my post.

4

Lots of young GC is expected, is it causing issues? Why do you think it's a leak?

5

yes, problem happens during these young GC, although these young GC looks pretty qucikly.
the problems is:
the read time out exception increase obviously during young GC, the exception are as follows:

java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at io.searchbox.client.http.JestHttpClient.executeRequest(JestHttpClient.java:109)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:56)

In my point of view, occasionally young GC is harmless, but so frequently young GC(5-10 seconds interval) may do have bad effect, because Both the young- and old-generation garbage collectors have phases that "stop the world."

6

Sometimes, the best way to have faster garbage collections is to give less memory to the JVM.

7

thanks, but what do you mean by saying that ? is 32g heap size to large? or is 12g young generation size too large?