# ES get Java heap space exception and can not startup correctly **URL:** \ **Category:** Elasticsearch\ **Created:** [August 29, 2013, 2:31am UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382 "2013-08-29T02:31:22Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![lijionly](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@lijionly](https://discuss.elastic.co/u/lijionly)\ **Post date:** [August 29, 2013, 2:31am UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382/1 "2013-08-29T02:31:22Z")
Hi, in elasticsearch.log, there are exceptions: [2013-08-29 16:52:29,789][WARN][transport.netty] [Union Jack] exception caught on transport layer [[id: 0x4cb1ed1c, /192.168.109.136:59970 =\> /192.168.109.135:9300]], closing connection java.lang.OutOfMemoryError: Java heap space [2013-08-29 16:52:35,857][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop. java.lang.OutOfMemoryError: Java heap space [2013-08-29 16:52:35,857][WARN][transport] [Union Jack] Received response for a request that has timed out, sent [569530ms] ago, timed out [536626ms] ago, action [discovery/zen/fd/masterPing], node [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561292] [2013-08-29 16:53:09,891][WARN][transport] [Union Jack] Received response for a request that has timed out, sent [570660ms] ago, timed out [515123ms] ago, action [discovery/zen/fd/masterPing], node [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561295] [2013-08-29 16:53:18,303][WARN][transport] [Union Jack] Received response for a request that has timed out, sent [519132ms] ago, timed out [474922ms] ago, action [discovery/zen/fd/masterPing], node [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561296] [2013-08-29 16:52:41,953][INFO][cluster.service] [Union Jack] detected\_master [Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]], added {[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]],}, reason: zen-disco-receive(from master [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]]) [2013-08-29 16:57:22,362][WARN][transport.netty] [Union Jack] exception caught on transport layer [[id: 0x5860f713, /192.168.109.136:59972 =\> /192.168.109.135:9300]], closing connection java.lang.OutOfMemoryError: Java heap space [2013-08-29 16:57:31,177][WARN][transport.netty] [Union Jack] exception caught on transport layer [[id: 0x4cb1ed1c, /192.168.109.136:59970 :\> /192.168.109.135:9300]], closing connection java.io.StreamCorruptedException: invalid internal transport message format at org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:27) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Then I try to restart Elasticsearch, and after it starts the log prints like this and the ES can not work: [2013-08-29 17:05:47,369][WARN][discovery.zen] [Prodigy] received a cluster state from [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part of the cluster, should not happen [2013-08-29 17:05:47,422][WARN][discovery.zen] [Prodigy] received a cluster state from [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part of the cluster, should not happen [2013-08-29 17:05:47,474][WARN][discovery.zen] [Prodigy] received a cluster state from [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part of the cluster, should not happen [2013-08-29 17:05:47,526][WARN][discovery.zen] [Prodigy] received a cluster state from [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part of the cluster, should not happen [2013-08-29 17:05:47,574][WARN][discovery.zen] [Prodigy] received a cluster state from [[Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part of the cluster, should not happen Why this happen and how to get back to normal? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![lijionly](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@lijionly](https://discuss.elastic.co/u/lijionly)\ **Post date:** [August 29, 2013, 2:33am UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382/2 "2013-08-29T02:33:07Z")
And I'm using the single node, no modified in elasticsearch.yml. All used are default configuration. 在 2013年8月29日星期四UTC+8上午10时31分22秒，liji...@gmail.com写道： > Hi, > > in elasticsearch.log, there are exceptions: > [2013-08-29 16:52:29,789][WARN][transport.netty] [Union Jack] > exception caught on transport layer [[id: 0x4cb1ed1c, / > 192.168.109.136:59970 =\> /192.168.109.135:9300]], closing connection > java.lang.OutOfMemoryError: Java heap space > [2013-08-29 16:52:35,857][WARN > ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the > selector loop. > java.lang.OutOfMemoryError: Java heap space > [2013-08-29 16:52:35,857][WARN][transport] [Union Jack] > Received response for a request that has timed out, sent [569530ms] ago, > timed out [536626ms] ago, action [discovery/zen/fd/masterPing], node [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561292] > [2013-08-29 16:53:09,891][WARN][transport] [Union Jack] > Received response for a request that has timed out, sent [570660ms] ago, > timed out [515123ms] ago, action [discovery/zen/fd/masterPing], node [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561295] > [2013-08-29 16:53:18,303][WARN][transport] [Union Jack] > Received response for a request that has timed out, sent [519132ms] ago, > timed out [474922ms] ago, action [discovery/zen/fd/masterPing], node [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561296] > [2013-08-29 16:52:41,953][INFO][cluster.service] [Union Jack] > detected\_master [Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]], added {[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]],}, reason: > zen-disco-receive(from master [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]]) > [2013-08-29 16:57:22,362][WARN][transport.netty] [Union Jack] > exception caught on transport layer [[id: 0x5860f713, / > 192.168.109.136:59972 =\> /192.168.109.135:9300]], closing connection > java.lang.OutOfMemoryError: Java heap space > [2013-08-29 16:57:31,177][WARN][transport.netty] [Union Jack] > exception caught on transport layer [[id: 0x4cb1ed1c, / > 192.168.109.136:59970 :\> /192.168.109.135:9300]], closing connection > java.io.StreamCorruptedException: invalid internal transport message format > at > org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:27) > at > org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) > at > org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) > at > org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) > at > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) > at > org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) > at > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > at > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) > at > org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) > at > org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) > at > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > at > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) > at > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) > at > org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) > at > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > > Then I try to restart Elasticsearch, and after it starts the log prints > like this and the ES can not work: > [2013-08-29 17:05:47,369][WARN][discovery.zen] [Prodigy] > received a cluster state from [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > of the cluster, should not happen > [2013-08-29 17:05:47,422][WARN][discovery.zen] [Prodigy] > received a cluster state from [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > of the cluster, should not happen > [2013-08-29 17:05:47,474][WARN][discovery.zen] [Prodigy] > received a cluster state from [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > of the cluster, should not happen > [2013-08-29 17:05:47,526][WARN][discovery.zen] [Prodigy] > received a cluster state from [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > of the cluster, should not happen > [2013-08-29 17:05:47,574][WARN][discovery.zen] [Prodigy] > received a cluster state from [[Big > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > of the cluster, should not happen > > Why this happen and how to get back to normal? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![lijionly](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@lijionly](https://discuss.elastic.co/u/lijionly)\ **Post date:** [August 29, 2013, 2:38am UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382/3 "2013-08-29T02:38:52Z")
the ES version is 0.90.2 在 2013年8月29日星期四UTC+8上午10时33分07秒，liji...@gmail.com写道： > And I'm using the single node, no modified in elasticsearch.yml. All used > are default configuration. > > 在 2013年8月29日星期四UTC+8上午10时31分22秒，liji...@gmail.com写道： > > > Hi, > > > > in elasticsearch.log, there are exceptions: > > [2013-08-29 16:52:29,789][WARN][transport.netty] [Union Jack] > > exception caught on transport layer [[id: 0x4cb1ed1c, / > > 192.168.109.136:59970 =\> /192.168.109.135:9300]], closing connection > > java.lang.OutOfMemoryError: Java heap space > > [2013-08-29 16:52:35,857][WARN > > ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the > > selector loop. > > java.lang.OutOfMemoryError: Java heap space > > [2013-08-29 16:52:35,857][WARN][transport] [Union Jack] > > Received response for a request that has timed out, sent [569530ms] ago, > > timed out [536626ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561292] > > [2013-08-29 16:53:09,891][WARN][transport] [Union Jack] > > Received response for a request that has timed out, sent [570660ms] ago, > > timed out [515123ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561295] > > [2013-08-29 16:53:18,303][WARN][transport] [Union Jack] > > Received response for a request that has timed out, sent [519132ms] ago, > > timed out [474922ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]], id [561296] > > [2013-08-29 16:52:41,953][INFO][cluster.service] [Union Jack] > > detected\_master [Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]], added {[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]],}, reason: > > zen-disco-receive(from master [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]]) > > [2013-08-29 16:57:22,362][WARN][transport.netty] [Union Jack] > > exception caught on transport layer [[id: 0x5860f713, / > > 192.168.109.136:59972 =\> /192.168.109.135:9300]], closing connection > > java.lang.OutOfMemoryError: Java heap space > > [2013-08-29 16:57:31,177][WARN][transport.netty] [Union Jack] > > exception caught on transport layer [[id: 0x4cb1ed1c, / > > 192.168.109.136:59970 :\> /192.168.109.135:9300]], closing connection > > java.io.StreamCorruptedException: invalid internal transport message > > format > > at > > org.elasticsearch.transport.netty.SizeHeaderFrameDecoder.decode(SizeHeaderFrameDecoder.java:27) > > at > > org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425) > > at > > org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) > > at > > org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) > > at > > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > > at > > org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) > > at > > org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74) > > at > > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) > > at > > org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) > > at > > org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268) > > at > > org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255) > > at > > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) > > at > > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) > > at > > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > > at > > org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) > > at > > org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) > > at > > org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) > > at > > org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > > > Then I try to restart Elasticsearch, and after it starts the log prints > > like this and the ES can not work: > > [2013-08-29 17:05:47,369][WARN][discovery.zen] [Prodigy] > > received a cluster state from [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > > of the cluster, should not happen > > [2013-08-29 17:05:47,422][WARN][discovery.zen] [Prodigy] > > received a cluster state from [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > > of the cluster, should not happen > > [2013-08-29 17:05:47,474][WARN][discovery.zen] [Prodigy] > > received a cluster state from [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > > of the cluster, should not happen > > [2013-08-29 17:05:47,526][WARN][discovery.zen] [Prodigy] > > received a cluster state from [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > > of the cluster, should not happen > > [2013-08-29 17:05:47,574][WARN][discovery.zen] [Prodigy] > > received a cluster state from [[Big > > Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/192.168.109.136:9300]]] and not part > > of the cluster, should not happen > > > > Why this happen and how to get back to normal? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [August 29, 2013, 7:08pm UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382/4 "2013-08-29T19:08:22Z")
It appears that there is more than one node. Either there is another instance of elasticsearch running on a different server, or you have two instances running on the same server. I see two IPs listed in the stack trace: 192.168.109.135 and 192.168.109.136 Cheers, Ivan On Wed, Aug 28, 2013 at 7:38 PM, [lijionly@gmail.com](mailto:lijionly@gmail.com) wrote: > the ES version is 0.90.2 > > 在 2013年8月29日星期四UTC+8上午10时33分07秒，liji...@gmail.com写道： > > > And I'm using the single node, no modified in elasticsearch.yml. All used > > are default configuration. > > > > 在 2013年8月29日星期四UTC+8上午10时31分22秒，\*\*liji...@gmail.com写道： > > > > > Hi, > > > > > > in elasticsearch.log, there are exceptions: > > > [2013-08-29 16:52:29,789][WARN][transport.netty] [Union Jack] > > > exception caught on transport layer [[id: 0x4cb1ed1c, / > > > 192.168.109.136:59970 =\> /192.168.109.135:9300]], closing connection > > > java.lang.OutOfMemoryError: Java heap space > > > [2013-08-29 16:52:35,857][WARN][netty.channel.socket.nio.\*\*AbstractNioSelector] > > > Unexpected exception in the selector loop. > > > java.lang.OutOfMemoryError: Java heap space > > > [2013-08-29 16:52:35,857][WARN][transport] [Union Jack] > > > Received response for a request that has timed out, sent [569530ms] ago, > > > timed out [536626ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > > Bertha][\*\*9NffWsOjTVmseFsoh2VtTA][inet[/\*\*192.168.109.136:9300]]], id > > > [561292] > > > [2013-08-29 16:53:09,891][WARN][transport] [Union Jack] > > > Received response for a request that has timed out, sent [570660ms] ago, > > > timed out [515123ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > > Bertha][\*\*9NffWsOjTVmseFsoh2VtTA][inet[/\*\*192.168.109.136:9300]]], id > > > [561295] > > > [2013-08-29 16:53:18,303][WARN][transport] [Union Jack] > > > Received response for a request that has timed out, sent [519132ms] ago, > > > timed out [474922ms] ago, action [discovery/zen/fd/masterPing], node [[Big > > > Bertha][\*\*9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]]], id > > > [561296] > > > [2013-08-29 16:52:41,953][INFO][cluster.service] [Union Jack] > > > detected\_master [Big Bertha][9NffWsOjTVmseFsoh2VtTA][inet[/ > > > 192.168.109.136:9300]], added {[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]],}, reason: > > > zen-disco-receive(from master [[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/\*\*192.168.109.136:9300]]]) > > > [2013-08-29 16:57:22,362][WARN][transport.netty] [Union Jack] > > > exception caught on transport layer [[id: 0x5860f713, / > > > 192.168.109.136:59972 =\> /192.168.109.135:9300]], closing connection > > > java.lang.OutOfMemoryError: Java heap space > > > [2013-08-29 16:57:31,177][WARN][transport.netty] [Union Jack] > > > exception caught on transport layer [[id: 0x4cb1ed1c, / > > > 192.168.109.136:59970 :\> /192.168.109.135:9300]], closing connection > > > [java.io](http://java.io).\*\*StreamCorruptedException: invalid internal transport message > > > format > > > at org.elasticsearch.transport. **netty.SizeHeaderFrameDecoder.** > > > decode(SizeHeaderFrameDecoder.\*\*java:27) > > > at org.elasticsearch.common. **netty.handler.codec.frame.** > > > FrameDecoder.callDecode(\*\*FrameDecoder.java:425) > > > at org.elasticsearch.common. **netty.handler.codec.frame.** > > > FrameDecoder.messageReceived(\*\*FrameDecoder.java:303) > > > at org.elasticsearch.common. **netty.channel.** > > > SimpleChannelUpstreamHandler.**handleUpstream(** > > > SimpleChannelUpstreamHandler.\*\*java:70) > > > at org.elasticsearch.common.\*\*netty.channel. **DefaultChannelPipeline.** > > > sendUpstream(\*\*DefaultChannelPipeline.java:\*\*564) > > > at org.elasticsearch.common.\*\*netty.channel. **DefaultChannelPipeline$** > > > DefaultChannelHandlerContext.**sendUpstream(** > > > DefaultChannelPipeline.java:\*\*791) > > > at org.elasticsearch.common. **netty.OpenChannelsHandler.** > > > handleUpstream(\*\*OpenChannelsHandler.java:74) > > > at org.elasticsearch.common.\*\*netty.channel. **DefaultChannelPipeline.** > > > sendUpstream(\*\*DefaultChannelPipeline.java:\*\*564) > > > at org.elasticsearch.common.\*\*netty.channel. **DefaultChannelPipeline.** > > > sendUpstream(\*\*DefaultChannelPipeline.java:\*\*559) > > > at org.elasticsearch.common. **netty.channel.Channels.** > > > fireMessageReceived(Channels.\*\*java:268) > > > at org.elasticsearch.common. **netty.channel.Channels.** > > > fireMessageReceived(Channels.\*\*java:255) > > > at org.elasticsearch.common. **netty.channel.socket.nio.** > > > NioWorker.read(NioWorker.java:\*\*88) > > > at org.elasticsearch.common. **netty.channel.socket.nio.** > > > AbstractNioWorker.process(\*\*AbstractNioWorker.java:109) > > > at org.elasticsearch.common. **netty.channel.socket.nio.** > > > AbstractNioSelector.run(\*\*AbstractNioSelector.java:312) > > > at org.elasticsearch.common. **netty.channel.socket.nio.** > > > AbstractNioWorker.run(\*\*AbstractNioWorker.java:90) > > > at org.elasticsearch.common. **netty.channel.socket.nio.** > > > NioWorker.run(NioWorker.java:\*\*178) > > > at org.elasticsearch.common.\*\*netty.util.**ThreadRenamingRunnable.run(** > > > ThreadRenamingRunnable.java:\*\*108) > > > at org.elasticsearch.common. **netty.util.internal.** > > > DeadLockProofWorker$1.run(\*\*DeadLockProofWorker.java:42) > > > at java.util.concurrent. **ThreadPoolExecutor$Worker.** > > > runTask(ThreadPoolExecutor.\*\*java:886) > > > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** > > > ThreadPoolExecutor.java:908) > > > at java.lang.Thread.run(Thread.\*\*java:662) > > > > > > Then I try to restart Elasticsearch, and after it starts the log prints > > > like this and the ES can not work: > > > [2013-08-29 17:05:47,369][WARN][discovery.zen] [Prodigy] > > > received a cluster state from [[Big Bertha][\*\* > > > 9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]]] and not part of > > > the cluster, should not happen > > > [2013-08-29 17:05:47,422][WARN][discovery.zen] [Prodigy] > > > received a cluster state from [[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]]] and not part of > > > the cluster, should not happen > > > [2013-08-29 17:05:47,474][WARN][discovery.zen] [Prodigy] > > > received a cluster state from [[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]]] and not part of > > > the cluster, should not happen > > > [2013-08-29 17:05:47,526][WARN][discovery.zen] [Prodigy] > > > received a cluster state from [[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/**192.168.109.136:9300]]] and not part of > > > the cluster, should not happen > > > [2013-08-29 17:05:47,574][WARN][discovery.zen] [Prodigy] > > > received a cluster state from [[Big Bertha][** > > > 9NffWsOjTVmseFsoh2VtTA][inet[/\*\*192.168.109.136:9300]]] and not part of > > > the cluster, should not happen > > > > > > Why this happen and how to get back to normal? > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:18am UTC](https://discuss.elastic.co/t/es-get-java-heap-space-exception-and-can-not-startup-correctly/13382/5 "2017-07-06T02:18:56Z")