Bulk API - curl: (55) Send failure: Connection was aborted


(LE) #1

Bulk API - curl: (55) Send failure: Connection was aborted

I've seen a lot of posts about this, but none of the solutions seem to work.
I've broken my bulk loads into separate file (~50Mb each).
The server a Xeon processor with 24GB RAM on Windows 2012R2.

If I run the curl command to bulk load the file, it works the first time.
The second file (same format) Fails with this error.

If I stop/restart the service, the 2nd file will work and the 3rd file will fail.

I've set the heap size to 12g/12g, but does nothing.

java.lang.OutOfMemoryError: Java heap space
at io.netty.buffer.PoolArena$HeapArena.newChunk(PoolArena.java:633) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:221) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.PoolArena.allocate(PoolArena.java:141) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:247) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:151) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:133) ~[netty-buffer-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73) ~[netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117) [netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610) [netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:513) [netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:467) [netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437) [netty-transport-4.1.5.Final.jar:4.1.5.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) [netty-common-4.1.5.Final.jar:4.1.5.Final]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]

Any other ideas.


(LE) #2

I finally found an solution that works on StackOverflow: http://stackoverflow.com/questions/27128620/elasticsearch-reports-default-heap-memory-size-after-setting-environment-variabl

Basically, I set the Heap size via Windows Environment variable: ES_HEAP_SIZE=12g
Then, I had to REMOVE the service and then re-INSTALL the service.


(system) #3