I have a problem for a while now, that I cannot find an explanation for it
so far.
I have a 10 nodes cluster with replication factor of 2 and 10 shards. On
each of the nodes, I have another JVM running my code, using transport
client, sending bulk indexing+delete requests. Sometimes, for a very small
number of the bulk requests, I get and response with errors for some of the
docs in the bulk with error "EsRejectedExecutionException". For these items
only, I create another bulk requests and sending them again. The problem is
that for some reason, a small number (something like 3000 out of 10,000,000
were not indexed and another 40 were not deleted) of the total docs are
"missing"; meaning that they are not indexed or deleted and I did not get
them marked as errors in the responses.
I log all errors and the only ones I get are those
"EsRejectedExecutionException" which I retry.
I see no errors in the ES log files.
I have a problem for a while now, that I cannot find an explanation for it
so far.
I have a 10 nodes cluster with replication factor of 2 and 10 shards. On
each of the nodes, I have another JVM running my code, using transport
client, sending bulk indexing+delete requests. Sometimes, for a very small
number of the bulk requests, I get and response with errors for some of the
docs in the bulk with error "EsRejectedExecutionException". For these items
only, I create another bulk requests and sending them again. The problem is
that for some reason, a small number (something like 3000 out of 10,000,000
were not indexed and another 40 were not deleted) of the total docs are
"missing"; meaning that they are not indexed or deleted and I did not get
them marked as errors in the responses.
I log all errors and the only ones I get are those
"EsRejectedExecutionException" which I retry.
I see no errors in the ES log files.
That exception means that the indexing/deleting queue is full thus these
documents are rejected.
I have a queue where I take the docs from and then send to ES for indexing.
all rejected are getting back into the queue. At the ends of the processes
the queue is empty !!
On Monday, May 19, 2014 11:50:16 PM UTC+3, Itamar Syn-Hershko wrote:
What are the details of that exception? can it be that ES has issues
parsing the docs?
On Mon, May 19, 2014 at 10:23 PM, Israel Klein <israel...@gmail.com<javascript:>
wrote:
Hi,
I have a problem for a while now, that I cannot find an explanation for
it so far.
I have a 10 nodes cluster with replication factor of 2 and 10 shards. On
each of the nodes, I have another JVM running my code, using transport
client, sending bulk indexing+delete requests. Sometimes, for a very small
number of the bulk requests, I get and response with errors for some of the
docs in the bulk with error "EsRejectedExecutionException". For these items
only, I create another bulk requests and sending them again. The problem is
that for some reason, a small number (something like 3000 out of 10,000,000
were not indexed and another 40 were not deleted) of the total docs are
"missing"; meaning that they are not indexed or deleted and I did not get
them marked as errors in the responses.
I log all errors and the only ones I get are those
"EsRejectedExecutionException" which I retry.
I see no errors in the ES log files.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.