OOM on every single call


(Maurício Linhares) #1

Hello guys,

I have just installed 0.18.5 on Lion and every single call I make
raises an OOM exception:

[2011-12-14 21:55:37,513][WARN ][transport.netty ] [Carter
Ghazikhanian] Exception caught on netty layer [[id: 0x718ddc48,
/0:0:0:0:0:0:0:1%0:60088 => /0:0:0:0:0:0:0:1%0:9300]]
java.lang.OutOfMemoryError: Java heap space
at org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelBuffer.java:47)
at org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(BigEndianHeapChannelBuffer.java:39)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:139)
at org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:73)
at org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicChannelBuffer.java:64)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:226)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:106)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:81)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:274)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:261)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)

I haven't done anything, this is a fresh install, command line looks
good on memory settings:

mauricio 17438 0,0 1,4 3789696 117124 s001 S 9:53
0:04.24 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xms256m -Xmx1g -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch
-Des.path.home=/Users/mauricio/desenvolvimento/elasticsearch-0.18.5
-cp :/Users/mauricio/desenvolvimento/elasticsearch-0.18.5/lib/:/Users/mauricio/desenvolvimento/elasticsearch-0.18.5/lib/sigar/
org.elasticsearch.bootstrap.ElasticSearch

Not sure what could be possibly happening in here.

Maurício Linhares
http://techbot.me/ - http://twitter.com/#!/mauriciojr


(Maurício Linhares-2) #2

Debugging I found that the callDecode() method on
MessageChannelHandler is saying the actual size for my request (which
is a GET) is 1195725860, which isn't correct. Looks like there is
something wrong here.

On Dec 14, 9:59 pm, Maurício Linhares mauricio.linha...@gmail.com
wrote:

Hello guys,

I have just installed 0.18.5 on Lion and every single call I make
raises an OOM exception:

[2011-12-14 21:55:37,513][WARN ][transport.netty ] [Carter
Ghazikhanian] Exception caught on netty layer [[id: 0x718ddc48,
/0:0:0:0:0:0:0:1%0:60088 => /0:0:0:0:0:0:0:1%0:9300]]
java.lang.OutOfMemoryError: Java heap space
at org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelB uffer.java:47)
at org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(Big EndianHeapChannelBuffer.java:39)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers. java:139)
at org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(He apChannelBufferFactory.java:73)
at org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicCh annelBuffer.java:64)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelB uffers.java:226)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(Mes sageChannelHandler.java:106)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleU pstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChanne lHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChann elsHandler.java:81)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:274)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:261)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker. java:351)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelected Keys(NioWorker.java:282)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.j ava:202)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenami ngRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(Dead LockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j ava:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: 908)
at java.lang.Thread.run(Thread.java:680)

I haven't done anything, this is a fresh install, command line looks
good on memory settings:

mauricio 17438 0,0 1,4 3789696 117124 s001 S 9:53
0:04.24 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xms256m -Xmx1g -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch
-Des.path.home=/Users/mauricio/desenvolvimento/elasticsearch-0.18.5
-cp :/Users/mauricio/desenvolvimento/elasticsearch-0.18.5/lib/:/Users/mauricio /desenvolvimento/elasticsearch-0.18.5/lib/sigar/
org.elasticsearch.bootstrap.ElasticSearch

Not sure what could be possibly happening in here.

Maurício Linhareshttp://techbot.me/-http://twitter.com/#!/mauriciojr


(Maurício Linhares-2) #3

Still on it, the code reads the first 4 bytes from the channel (as if
this was some kind of message) to obtain the file size, which is
wrong, since this is an HTTP request and it doesn't write the request
size in it's first 4 bytes.

Not sure why this Netty request handler is coming in here.

Anyone has any clue on what's happening here?

On Dec 14, 10:38 pm, Maurício Linhares linhares.mauri...@gmail.com
wrote:

Debugging I found that the callDecode() method on
MessageChannelHandler is saying the actual size for my request (which
is a GET) is 1195725860, which isn't correct. Looks like there is
something wrong here.

On Dec 14, 9:59 pm, Maurício Linhares mauricio.linha...@gmail.com
wrote:

Hello guys,

I have just installed 0.18.5 on Lion and every single call I make
raises an OOM exception:

[2011-12-14 21:55:37,513][WARN ][transport.netty ] [Carter
Ghazikhanian] Exception caught on netty layer [[id: 0x718ddc48,
/0:0:0:0:0:0:0:1%0:60088 => /0:0:0:0:0:0:0:1%0:9300]]
java.lang.OutOfMemoryError: Java heap space
at org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelB uffer.java:47)
at org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(Big EndianHeapChannelBuffer.java:39)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers. java:139)
at org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(He apChannelBufferFactory.java:73)
at org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicCh annelBuffer.java:64)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelB uffers.java:226)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(Mes sageChannelHandler.java:106)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleU pstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChanne lHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChann elsHandler.java:81)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:274)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:261)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker. java:351)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelected Keys(NioWorker.java:282)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.j ava:202)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenami ngRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(Dead LockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j ava:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: 908)
at java.lang.Thread.run(Thread.java:680)

I haven't done anything, this is a fresh install, command line looks
good on memory settings:

mauricio 17438 0,0 1,4 3789696 117124 s001 S 9:53
0:04.24 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xms256m -Xmx1g -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch
-Des.path.home=/Users/mauricio/desenvolvimento/elasticsearch-0.18.5
-cp :/Users/mauricio/desenvolvimento/elasticsearch-0.18.5/lib/:/Users/mauricio /desenvolvimento/elasticsearch-0.18.5/lib/sigar/
org.elasticsearch.bootstrap.ElasticSearch

Not sure what could be possibly happening in here.

Maurício Linhareshttp://techbot.me/-http://twitter.com/#!/mauriciojr


(Maurício Linhares-2) #4

And i was requesting at the wrong port, sorry for the inconvenience.

Maurício Linhares
http://techbot.me/ - http://twitter.com/#!/mauriciojr

2011/12/14 Maurício Linhares linhares.mauricio@gmail.com:

Still on it, the code reads the first 4 bytes from the channel (as if
this was some kind of message) to obtain the file size, which is
wrong, since this is an HTTP request and it doesn't write the request
size in it's first 4 bytes.

Not sure why this Netty request handler is coming in here.

Anyone has any clue on what's happening here?

On Dec 14, 10:38 pm, Maurício Linhares linhares.mauri...@gmail.com
wrote:

Debugging I found that the callDecode() method on
MessageChannelHandler is saying the actual size for my request (which
is a GET) is 1195725860, which isn't correct. Looks like there is
something wrong here.

On Dec 14, 9:59 pm, Maurício Linhares mauricio.linha...@gmail.com
wrote:

Hello guys,

I have just installed 0.18.5 on Lion and every single call I make
raises an OOM exception:

[2011-12-14 21:55:37,513][WARN ][transport.netty ] [Carter
Ghazikhanian] Exception caught on netty layer [[id: 0x718ddc48,
/0:0:0:0:0:0:0:1%0:60088 => /0:0:0:0:0:0:0:1%0:9300]]
java.lang.OutOfMemoryError: Java heap space
at org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelB uffer.java:47)
at org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(Big EndianHeapChannelBuffer.java:39)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers. java:139)
at org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(He apChannelBufferFactory.java:73)
at org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicCh annelBuffer.java:64)
at org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelB uffers.java:226)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(Mes sageChannelHandler.java:106)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleU pstream(SimpleChannelUpstreamHandler.java:80)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChanne lHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChann elsHandler.java:81)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream( DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:274)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channel s.java:261)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker. java:351)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.processSelected Keys(NioWorker.java:282)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.j ava:202)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenami ngRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(Dead LockProofWorker.java:44)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.j ava:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: 908)
at java.lang.Thread.run(Thread.java:680)

I haven't done anything, this is a fresh install, command line looks
good on memory settings:

mauricio 17438 0,0 1,4 3789696 117124 s001 S 9:53
0:04.24 /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java
-Xms256m -Xmx1g -Xss128k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
-XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch
-Des.path.home=/Users/mauricio/desenvolvimento/elasticsearch-0.18.5
-cp :/Users/mauricio/desenvolvimento/elasticsearch-0.18.5/lib/:/Users/mauricio /desenvolvimento/elasticsearch-0.18.5/lib/sigar/
org.elasticsearch.bootstrap.ElasticSearch

Not sure what could be possibly happening in here.

Maurício Linhareshttp://techbot.me/-http://twitter.com/#!/mauriciojr


(system) #5