Hi,
I was testing async indexing to an Elastic Search Node via java
transport api , the ES node have actually 1GB of memory via settings -
Xms1G^ -Xmx1G^ .
ES Node was running on Windows XP ( same problem occurs at Linux
also ) .
The test was like this :
Indexing objects to ES in async mode , objects have only name and
surname .
When i index 50K object everything is fine,
When i index 100K objects i see the OutOfMemoryError in the logs,
It seems on some limit the error occurs.
When the error occurs the heap size is around 250MB , means there is
still 750MB free memory to use.
I did some research and found that the reason may be the stack size
from http://candrews.integralblue.com/2009/01/preventing-outofmemoryerror-native-thread/
For 100K objects I have changed elasticsearch.bat tried -Xss256k ,
sometimes it gives no error but sometimes the error occurs again.
Please advise on this: is this a bug, if this is not a bug how should
we overcome the problem.
The error from ES logs :
[18:26:53,392][WARN ][transport.netty ] [Immortus] Exception
caught on netty layer [[id: 0x01bca1c3, /192.168.1.107:27288 => /
192.168.1.107:9350]] java.lang.OutOfMemoryError: unable to create new
native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at
java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:
727)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:
657)
at
org.elasticsearch.threadpool.support.AbstractThreadPool.execute(AbstractThreadPool.java:
146)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction
$AsyncShardOperationAction.start(TransportShardReplicationOperationAction.java:
276)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction
$AsyncShardOperationAction.start(TransportShardReplicationOperationAction.java:
240)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction.doExecute(TransportShardReplicationOperationAction.java:
93)
at
org.elasticsearch.action.index.TransportIndexAction.doExecute(TransportIndexAction.java:
101)
at
org.elasticsearch.action.index.TransportIndexAction.doExecute(TransportIndexAction.java:
57)
at
org.elasticsearch.action.support.BaseAction.execute(BaseAction.java:
54)
...
Regards,
Abdurrahim Eke
www.ifountain.com