I am using ES to search a couchdb. i am running it on a EC2 Mirco instance, ES v0.17.6, java v1.6. i am running only one node for a protoype before i start the actual product. I have encountered an issue when setting this up.
The issue that i encounter, is that while indexing, the CPU Spikes to to 100% and brings my server to crawl. I am indexing only about 192k docs, but this indexing has never actually finished. This CPU spikes sometimes gets so high that the server crashes, and when i restart ES, the river that i made is corrupted.
Any guidance to why this is happening?
i see error messages in the logs and then happen alot......
[2011-12-10 19:47:28,034][WARN ][river.couchdb ] [Sultan] [couchdb][lr] failed to executefailure in bulk execution:
[2]: index [lr], type [lr], id [c6f81aec0e6387e0badc62c3d3c8c591], message [IndexFailedEngineException[[lr][4] Index failed for [lr#c6f81aec0e6387e0badc62c3d3c8c591]]; nested: IOException[No space left on device]; ]
-and-
org.elasticsearch.index.engine.IndexFailedEngineException: [lr][1] Index failed for [lr#c6f81aec0e6387e0badc62c3d3c8f17f]
at org.elasticsearch.index.engine.robin.RobinEngine.index(RobinEngine.java:474)
at org.elasticsearch.index.shard.service.InternalIndexShard.index(InternalIndexShard.java:303)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:131)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:428)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:341)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.io.IOException: No space left on device
at java.io.RandomAccessFile.writeBytes(Native Method)
at java.io.RandomAccessFile.write(RandomAccessFile.java:499)
at org.apache.lucene.store.FSDirectory$FSIndexOutput.flushBuffer(FSDirectory.java:448)
at org.apache.lucene.store.BufferedIndexOutput.flushBuffer(BufferedIndexOutput.java:99)
at org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:88)
at org.apache.lucene.store.BufferedIndexOutput.writeBytes(BufferedIndexOutput.java:77)
at org.elasticsearch.index.store.support.AbstractStore$StoreIndexOutput.writeBytes(AbstractStore.java:470)
at org.apache.lucene.store.DataOutput.writeBytes(DataOutput.java:43)
at org.apache.lucene.store.RAMOutputStream.writeTo(RAMOutputStream.java:65)
at org.apache.lucene.index.FieldsWriter.flushDocument(FieldsWriter.java:116)
at org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:113)
at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:152)
at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1398)
at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1418)
at org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1037)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:772)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2056)
at org.elasticsearch.index.engine.robin.RobinEngine.innerIndex(RobinEngine.java:561)
at org.elasticsearch.index.engine.robin.RobinEngine.index(RobinEngine.java:469)
... 7 more