# Missing documents after bulk indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [May 19, 2014, 7:23pm UTC](https://discuss.elastic.co/t/missing-documents-after-bulk-indexing/17601 "2014-05-19T19:23:18Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![israel](https://avatars.discourse-cdn.com/v4/letter/i/b77776/32.png) [@israel](https://discuss.elastic.co/u/israel)\ **Post date:** [May 19, 2014, 7:23pm UTC](https://discuss.elastic.co/t/missing-documents-after-bulk-indexing/17601/1 "2014-05-19T19:23:18Z")
Hi, I have a problem for a while now, that I cannot find an explanation for it so far. I have a 10 nodes cluster with replication factor of 2 and 10 shards. On each of the nodes, I have another JVM running my code, using transport client, sending bulk indexing+delete requests. Sometimes, for a very small number of the bulk requests, I get and response with errors for some of the docs in the bulk with error "EsRejectedExecutionException". For these items only, I create another bulk requests and sending them again. The problem is that for some reason, a small number (something like 3000 out of 10,000,000 were not indexed and another 40 were not deleted) of the total docs are "missing"; meaning that they are not indexed or deleted and I did not get them marked as errors in the responses. I log all errors and the only ones I get are those "EsRejectedExecutionException" which I retry. I see no errors in the ES log files. Thanks, Israel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [May 19, 2014, 8:50pm UTC](https://discuss.elastic.co/t/missing-documents-after-bulk-indexing/17601/2 "2014-05-19T20:50:16Z")
What are the details of that exception? can it be that ES has issues parsing the docs? -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Mon, May 19, 2014 at 10:23 PM, Israel Klein [israel.klein@gmail.com](mailto:israel.klein@gmail.com)wrote: > Hi, > > I have a problem for a while now, that I cannot find an explanation for it > so far. > I have a 10 nodes cluster with replication factor of 2 and 10 shards. On > each of the nodes, I have another JVM running my code, using transport > client, sending bulk indexing+delete requests. Sometimes, for a very small > number of the bulk requests, I get and response with errors for some of the > docs in the bulk with error "EsRejectedExecutionException". For these items > only, I create another bulk requests and sending them again. The problem is > that for some reason, a small number (something like 3000 out of 10,000,000 > were not indexed and another 40 were not deleted) of the total docs are > "missing"; meaning that they are not indexed or deleted and I did not get > them marked as errors in the responses. > I log all errors and the only ones I get are those > "EsRejectedExecutionException" which I retry. > I see no errors in the ES log files. > > Thanks, > > Israel > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtxUEYG-fBEqXprNFf97JkEGYQBFQ97zZ0DTmJRq4fX-g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtxUEYG-fBEqXprNFf97JkEGYQBFQ97zZ0DTmJRq4fX-g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![israel](https://avatars.discourse-cdn.com/v4/letter/i/b77776/32.png) [@israel](https://discuss.elastic.co/u/israel)\ **Post date:** [May 19, 2014, 9:25pm UTC](https://discuss.elastic.co/t/missing-documents-after-bulk-indexing/17601/3 "2014-05-19T21:25:02Z")
That exception means that the indexing/deleting queue is full thus these documents are rejected. I have a queue where I take the docs from and then send to ES for indexing. all rejected are getting back into the queue. At the ends of the processes the queue is empty !! On Monday, May 19, 2014 11:50:16 PM UTC+3, Itamar Syn-Hershko wrote: > What are the details of that exception? can it be that ES has issues > parsing the docs? > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > On Mon, May 19, 2014 at 10:23 PM, Israel Klein \<[israel...@gmail.com](mailto:israel...@gmail.com)\ > > > wrote: > > > Hi, > > > > I have a problem for a while now, that I cannot find an explanation for > > it so far. > > I have a 10 nodes cluster with replication factor of 2 and 10 shards. On > > each of the nodes, I have another JVM running my code, using transport > > client, sending bulk indexing+delete requests. Sometimes, for a very small > > number of the bulk requests, I get and response with errors for some of the > > docs in the bulk with error "EsRejectedExecutionException". For these items > > only, I create another bulk requests and sending them again. The problem is > > that for some reason, a small number (something like 3000 out of 10,000,000 > > were not indexed and another 40 were not deleted) of the total docs are > > "missing"; meaning that they are not indexed or deleted and I did not get > > them marked as errors in the responses. > > I log all errors and the only ones I get are those > > "EsRejectedExecutionException" which I retry. > > I see no errors in the ES log files. > > > > Thanks, > > > > Israel > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8e9f67d1-3842-44b2-8598-645ce233f6b6%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/23ad3027-ec2c-4023-9176-636596a39b2c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/23ad3027-ec2c-4023-9176-636596a39b2c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:28am UTC](https://discuss.elastic.co/t/missing-documents-after-bulk-indexing/17601/4 "2017-07-06T01:28:22Z")