HI. When I use elasticsearch-hadoop, I encounter this error：
Could not write all entries [1576/10485504
https://github.com/elasticsearch/elasticsearch-hadoop/commit/maybe%20ES%20was%20overloaded?].
Bailing out...
My task execution schedule is as follows：
14/11/05 16:06:06 INFO mapred.JobClient: map 100% reduce 69%
14/11/05 16:06:12 INFO mapred.JobClient: map 100% reduce 66%
14/11/05 16:06:14 INFO mapred.JobClient: map 100% reduce 65%
14/11/05 16:06:16 INFO mapred.JobClient: map 100% reduce 66%
14/11/05 16:06:20 INFO mapred.JobClient: map 100% reduce 67%
14/11/05 16:06:49 INFO mapred.JobClient: map 100% reduce 68%
14/11/05 16:06:57 INFO mapred.JobClient: map 100% reduce 69%
14/11/05 16:07:09 INFO mapred.JobClient: map 100% reduce 70%
14/11/05 16:08:21 INFO mapred.JobClient: map 100% reduce 69%
14/11/05 16:08:25 INFO mapred.JobClient: map 100% reduce 68%
14/11/05 16:08:33 INFO mapred.JobClient: map 100% reduce 69%
14/11/05 16:09:08 INFO mapred.JobClient: map 100% reduce 70%
14/11/05 16:10:02 INFO mapred.JobClient: map 100% reduce 69%
14/11/05 16:10:04 INFO mapred.JobClient: map 100% reduce 68%
14/11/05 16:10:06 INFO mapred.JobClient: map 100% reduce 65%
14/11/05 16:10:09 INFO mapred.JobClient: map 100% reduce 62%
14/11/05 16:10:11 INFO mapred.JobClient: map 100% reduce 61%
14/11/05 16:10:13 INFO mapred.JobClient: map 100% reduce 62%
14/11/05 16:10:15 INFO mapred.JobClient: map 100% reduce 63%
14/11/05 16:10:17 INFO mapred.JobClient: map 100% reduce 64%
14/11/05 16:10:19 INFO mapred.JobClient: map 100% reduce 65%
14/11/05 16:10:25 INFO mapred.JobClient: map 100% reduce 64%
14/11/05 16:10:35 INFO mapred.JobClient: map 100% reduce 65%
14/11/05 16:10:45 INFO mapred.JobClient: map 100% reduce 66%
14/11/05 16:10:49 INFO mapred.JobClient: map 100% reduce 67%
14/11/05 16:10:51 INFO mapred.JobClient: map 100% reduce 68%
14/11/05 16:10:59 INFO mapred.JobClient: map 100% reduce 69%
My parameters are set as follows:
conf.set("es.batch.size.bytes", "1048576");
conf.set("es.batch.size.entries", "1000");
conf.set("mapred.reduce.tasks", "200");
conf.setBoolean("mapred.map.tasks.speculative.execution", false);
conf.setBoolean("mapred.reduce.tasks.speculative.execution", false);
The jar I used is: elasticsearch-hadoop-2.1.0.Beta2.jar
My hardware configuration is as follows：
100GB RAM , 4TB hard disk, 10 nodes, 1 balancer, 1 master
Can anyone help me find out the reason why this error appear? Thanks
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/80420872-9b3e-42ac-afe7-386bc37f6664%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.