# Error on the network layer **URL:** \ **Category:** Elasticsearch\ **Created:** [December 7, 2016, 6:47pm UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345 "2016-12-07T18:47:33Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![\_117228](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/_117228/32/14522_2.png) [@\_117228](https://discuss.elastic.co/u/_117228)\ **Post date:** [December 7, 2016, 6:47pm UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/1 "2016-12-07T18:47:33Z")
I have a problem with a single node elasticsearch instance on a m4.4xlarge with an EBS volume attached and heap size of 8 GB. Every now and then my elasticsearch service stops with these errors in the logs : ```auto at org.elasticsearch.transport.netty4.Netty4Utils.maybeDie(Netty4Utils.java:138) at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.exceptionCaught(Netty4HttpRequestHandler.java:76) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:274) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:266) . . . ``` and right after with : ```auto java.lang.OutOfMemoryError: Java heap space at io.netty.buffer.PoolArena$HeapArena.newChunk(PoolArena.java:633) ~[?:?] at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237) ~[?:?] at io.netty.buffer.PoolArena.allocate(PoolArena.java:221) ~[?:?] at io.netty.buffer.PoolArena.allocate(PoolArena.java:141) ~[?:?] . . . ``` But I'm sure it's running with 8GB heap space then technically it shouldn't be out of memory. Searching for the first network issue results in nothing ... looks like not many people had that issue. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 8, 2016, 1:48am UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/2 "2016-12-08T01:48:38Z")
What version, what OS and version, what JVM and version? ---
**Author:** ![\_117228](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/_117228/32/14522_2.png) [@\_117228](https://discuss.elastic.co/u/_117228)\ **Post date:** [December 8, 2016, 3:21am UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/3 "2016-12-08T03:21:47Z")
Hi! Elasticsearch 5 on Amazon AMI (`4.4.23-31.54.amzn1.x86_64`) and java version "1.7.0\_121" OpenJDK Runtime Environment (amzn-2.6.8.1.69.amzn1-x86\_64 u121-b00) OpenJDK 64-Bit Server VM (build 24.121-b00, mixed mode) I've increased the heap size to 32G. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 8, 2016, 10:14am UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/4 "2016-12-08T10:14:52Z")
According to the [support matrix](https://www.elastic.co/support/matrix#show_jvm), Elasticsearch 5.0 requires Java 8, so you probably should upgrade that. You should also try to keep the heap size a bit below 32GB in order to be able to benefit from compressed pointers. I think 30 GB should be OK, but it may be possible to push it a bit higher. ---
**Author:** ![\_117228](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/_117228/32/14522_2.png) [@\_117228](https://discuss.elastic.co/u/_117228)\ **Post date:** [December 8, 2016, 3:31pm UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/5 "2016-12-08T15:31:51Z")
Noted. My box has 64 GB of memory and I've given Elasticsearch 32 GB of heap. This problem happened again. ps aux | grep elasticsearch gives me : ``/usr/lib/jvm/jre-1.8.0/bin/java -Xms32g -Xmx32g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Xms8g -Xmx8g -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-5.0.2.jar:/usr/share/elasticsearch/lib/\* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid -d -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/data/elasticsearch -Edefault.path.conf=/etc/elasticsearch ```auto ``` ---
**Author:** ![\_117228](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/_117228/32/14522_2.png) [@\_117228](https://discuss.elastic.co/u/_117228)\ **Post date:** [December 8, 2016, 4:01pm UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/6 "2016-12-08T16:01:22Z")
Hi! Elasticsearch 5 on Amazon AMI (4.4.23-31.54.amzn1.x86\_64) and java version "1.7.0\_121" OpenJDK Runtime Environment (amzn-2.6.8.1.69.amzn1-x86\_64 u121-b00) OpenJDK 64-Bit Server VM (build 24.121-b00, mixed mode) I've increased the heap size to 32G. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 5, 2017, 4:01pm UTC](https://discuss.elastic.co/t/error-on-the-network-layer/68345/7 "2017-01-05T16:01:22Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.