# Max documents 10,500? **URL:** \ **Category:** Elasticsearch\ **Created:** [May 1, 2015, 4:36pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514 "2015-05-01T16:36:24Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![blake1024](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@blake1024](https://discuss.elastic.co/u/blake1024)\ **Post date:** [May 1, 2015, 4:36pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/1 "2015-05-01T16:36:24Z")
Greetings, I have two similar but unrelated machines. I am adding 50,000+ documents to each. Afterwards, one shows the 50,000+ documents and the other only shows 10,500. The second machine seems to be capping out at 10,500. Why, and how can I correct this? The relevant facts are as follows: 1. Both machines are current 64 bit Linux machines with at least 8GB of RAM and more than sufficient disk space. 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is running local to each machine. 3. Both machines are running the exact same program to load up ES. Each has nearly default ES config files (just different names). 4. The program keeps a counter of the number of times documents are added to ES, and the return codes of each add is checked. Both are 50,000+. 5. When I do a the same query on each machine with curl, the good machine shows a max\_score of 8.2, the bad machine shows .499 - remember, same set of documents and same search query. I've spent a day on this, and I am running out of ideas. Any help would sure be appreciated. Blake McBride -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 1, 2015, 4:41pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/2 "2015-05-01T16:41:57Z")
If you have nothing in logs it could mean that you have an issue with your injector. May be you are using bulk but you don't check the bulk response? David > Le 1 mai 2015 à 18:36, Blake McBride [blake1024@gmail.com](mailto:blake1024@gmail.com) a écrit : > > Greetings, > > I have two similar but unrelated machines. I am adding 50,000+ documents to each. Afterwards, one shows the 50,000+ documents and the other only shows 10,500. The second machine seems to be capping out at 10,500. Why, and how can I correct this? The relevant facts are as follows: > > 1. Both machines are current 64 bit Linux machines with at least 8GB of RAM and more than sufficient disk space. > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is running local to each machine. > > 3. Both machines are running the exact same program to load up ES. Each has nearly default ES config files (just different names). > > 4. The program keeps a counter of the number of times documents are added to ES, and the return codes of each add is checked. Both are 50,000+. > > 5. When I do a the same query on each machine with curl, the good machine shows a max\_score of 8.2, the bad machine shows .499 - remember, same set of documents and same search query. > > I've spent a day on this, and I am running out of ideas. Any help would sure be appreciated. > > Blake McBride > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/E2B14086-CFD2-4DF0-AC4A-E00A0204C8A5%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/E2B14086-CFD2-4DF0-AC4A-E00A0204C8A5%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![blake1024](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@blake1024](https://discuss.elastic.co/u/blake1024)\ **Post date:** [May 1, 2015, 6:40pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/3 "2015-05-01T18:40:43Z")
The log only contains: [2015-05-01 18:22:10,398][INFO][cluster.metadata] [mmsapp-na-component] [components-1430504530354] creating index, cause [api], templates , shards [5]/[1], mappings [index\_name, component] Each document is being added individually from JavaScript via: exports.addDoc = function (index, type, id, doc, callback) { if (client !== undefined) { var json = { index: index, type: type, id: id, body: doc }; client.create(json, callback); } else if (callback !== undefined) { callback('Elasticsearch not connected', undefined); } }; On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > If you have nothing in logs it could mean that you have an issue with your > injector. > May be you are using bulk but you don't check the bulk response? > > David > > Le 1 mai 2015 à 18:36, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a > écrit : > > Greetings, > > I have two similar but unrelated machines. I am adding 50,000+ documents > to each. Afterwards, one shows the 50,000+ documents and the other only > shows 10,500. The second machine seems to be capping out at 10,500. Why, > and how can I correct this? The relevant facts are as follows: > > 1. Both machines are current 64 bit Linux machines with at least 8GB of > RAM and more than sufficient disk space. > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is > running local to each machine. > > 3. Both machines are running the exact same program to load up ES. Each > has nearly default ES config files (just different names). > > 4. The program keeps a counter of the number of times documents are added > to ES, and the return codes of each add is checked. Both are 50,000+. > > 5. When I do a the same query on each machine with curl, the good machine > shows a max\_score of 8.2, the bad machine shows .499 - remember, same set > of documents and same search query. > > I've spent a day on this, and I am running out of ideas. Any help would > sure be appreciated. > > Blake McBride > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 1, 2015, 6:57pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/4 "2015-05-01T18:57:24Z")
Could you add a counter in your JS app to make sure you sent all docs? I suspect something wrong in your index process -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 1 mai 2015 à 20:40, Blake McBride [blake1024@gmail.com](mailto:blake1024@gmail.com) a écrit : > > The log only contains: > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] [mmsapp-na-component] [components-1430504530354] creating index, cause [api], templates , shards [5]/[1], mappings [index\_name, component] > > Each document is being added individually from JavaScript via: > > exports.addDoc = function (index, type, id, doc, callback) { > if (client !== undefined) { > var json = { > index: index, > type: type, > id: id, > body: doc > }; > client.create(json, callback); > } else if (callback !== undefined) { > callback('Elasticsearch not connected', undefined); > } > }; > > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > If you have nothing in logs it could mean that you have an issue with your injector. > > May be you are using bulk but you don't check the bulk response? > > > > David > > > > > Le 1 mai 2015 à 18:36, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > > > Greetings, > > > > > > I have two similar but unrelated machines. I am adding 50,000+ documents to each. Afterwards, one shows the 50,000+ documents and the other only shows 10,500. The second machine seems to be capping out at 10,500. Why, and how can I correct this? The relevant facts are as follows: > > > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB of RAM and more than sufficient disk space. > > > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is running local to each machine. > > > > > > 3. Both machines are running the exact same program to load up ES. Each has nearly default ES config files (just different names). > > > > > > 4. The program keeps a counter of the number of times documents are added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > > > 5. When I do a the same query on each machine with curl, the good machine shows a max\_score of 8.2, the bad machine shows .499 - remember, same set of documents and same search query. > > > > > > I've spent a day on this, and I am running out of ideas. Any help would sure be appreciated. > > > > > > Blake McBride > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/137922C9-B539-452E-9181-B82E16AFC9C5%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/137922C9-B539-452E-9181-B82E16AFC9C5%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![blake1024](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@blake1024](https://discuss.elastic.co/u/blake1024)\ **Post date:** [May 1, 2015, 7:25pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/5 "2015-05-01T19:25:50Z")
I changed the code to read: var counter = 0; exports.addDoc = function (index, type, id, doc, callback) { if (client !== undefined) { var json = { index: index, type: type, id: id, body: doc }; if (counter++ % 1000 === 0) { console.log('Adding document #' + counter); } client.create(json, callback); } else if (callback !== undefined) { callback('Elasticsearch not connected', undefined); } }; The last printout reads: Adding document #53001 The code that does the error check looks like: esUtils.addDoc(esIndex, 'component', compObj.id, doc, function (err, response, status) { if (err !== undefined || status !== 201 || response.created !== true) { console.log('Unexpected ES response: ' + status + ' ' + err + response.created); } }); I never see that message. Finally, after the above I get: $ curl -s -XPOST '[http://localhost:9200/components/\_count](http://localhost:9200/components/_count)' {"count":10500,"\_shards":{"total":5,"successful":5,"failed":0}} Thanks for the help! On Friday, May 1, 2015 at 1:57:42 PM UTC-5, David Pilato wrote: > Could you add a counter in your JS app to make sure you sent all docs? > > I suspect something wrong in your index process > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 1 mai 2015 à 20:40, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a > écrit : > > The log only contains: > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] > [mmsapp-na-component] [components-1430504530354] creating index, cause > [api], templates , shards [5]/[1], mappings [index\_name, component] > > Each document is being added individually from JavaScript via: > > exports.addDoc = function (index, type, id, doc, callback) { > if (client !== undefined) { > var json = { > index: index, > type: type, > id: id, > body: doc > }; > client.create(json, callback); > } else if (callback !== undefined) { > callback('Elasticsearch not connected', undefined); > } > }; > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > > If you have nothing in logs it could mean that you have an issue with > > your injector. > > May be you are using bulk but you don't check the bulk response? > > > > David > > > > Le 1 mai 2015 à 18:36, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > Greetings, > > > > I have two similar but unrelated machines. I am adding 50,000+ documents > > to each. Afterwards, one shows the 50,000+ documents and the other only > > shows 10,500. The second machine seems to be capping out at 10,500. Why, > > and how can I correct this? The relevant facts are as follows: > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB of > > RAM and more than sufficient disk space. > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is > > running local to each machine. > > > > 3. Both machines are running the exact same program to load up ES. Each > > has nearly default ES config files (just different names). > > > > 4. The program keeps a counter of the number of times documents are > > added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > 5. When I do a the same query on each machine with curl, the good > > machine shows a max\_score of 8.2, the bad machine shows .499 - remember, > > same set of documents and same search query. > > > > I've spent a day on this, and I am running out of ideas. Any help would > > sure be appreciated. > > > > Blake McBride > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 1, 2015, 7:50pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/6 "2015-05-01T19:50:47Z")
Any chance you are using the same id multiple times? -- David Pilato - Developer | Evangelist > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co/)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > Le 1 mai 2015 à 21:25, Blake McBride [blake1024@gmail.com](mailto:blake1024@gmail.com) a écrit : > > I changed the code to read: > > var counter = 0; > > exports.addDoc = function (index, type, id, doc, callback) { > if (client !== undefined) { > var json = { > index: index, > type: type, > id: id, > body: doc > }; > if (counter++ % 1000 === 0) { > console.log('Adding document #' + counter); > } > client.create(json, callback); > } else if (callback !== undefined) { > callback('Elasticsearch not connected', undefined); > } > }; > > The last printout reads: Adding document #53001 > > The code that does the error check looks like: > > esUtils.addDoc(esIndex, 'component', compObj.id, doc, function (err, response, status) { > if (err !== undefined || status !== 201 || response.created !== true) { > console.log('Unexpected ES response: ' + status + ' ' + err + response.created); > } > }); > > I never see that message. Finally, after the above I get: > > $ curl -s -XPOST '[http://localhost:9200/components/\_count](http://localhost:9200/components/_count)' > {"count":10500,"\_shards":{"total":5,"successful":5,"failed":0}} > > Thanks for the help! > > On Friday, May 1, 2015 at 1:57:42 PM UTC-5, David Pilato wrote: > Could you add a counter in your JS app to make sure you sent all docs? > > I suspect something wrong in your index process > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 1 mai 2015 à 20:40, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a écrit : > > > The log only contains: > > > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] [mmsapp-na-component] [components-1430504530354] creating index, cause [api], templates , shards [5]/[1], mappings [index\_name, component] > > > > Each document is being added individually from JavaScript via: > > > > exports.addDoc = function (index, type, id, doc, callback) { > > if (client !== undefined) { > > var json = { > > index: index, > > type: type, > > id: id, > > body: doc > > }; > > client.create(json, callback); > > } else if (callback !== undefined) { > > callback('Elasticsearch not connected', undefined); > > } > > }; > > > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > If you have nothing in logs it could mean that you have an issue with your injector. > > May be you are using bulk but you don't check the bulk response? > > > > David > > > > Le 1 mai 2015 à 18:36, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \<\>\> a écrit : > > > > > Greetings, > > > > > > I have two similar but unrelated machines. I am adding 50,000+ documents to each. Afterwards, one shows the 50,000+ documents and the other only shows 10,500. The second machine seems to be capping out at 10,500. Why, and how can I correct this? The relevant facts are as follows: > > > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB of RAM and more than sufficient disk space. > > > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is running local to each machine. > > > > > > 3. Both machines are running the exact same program to load up ES. Each has nearly default ES config files (just different names). > > > > > > 4. The program keeps a counter of the number of times documents are added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > > > 5. When I do a the same query on each machine with curl, the good machine shows a max\_score of 8.2, the bad machine shows .499 - remember, same set of documents and same search query. > > > > > > I've spent a day on this, and I am running out of ideas. Any help would sure be appreciated. > > > > > > Blake McBride > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<\>. > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm_medium=email&utm_source=footer). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/51F802A3-4952-4A8D-B322-1547E9B88668%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/51F802A3-4952-4A8D-B322-1547E9B88668%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![blake1024](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@blake1024](https://discuss.elastic.co/u/blake1024)\ **Post date:** [May 1, 2015, 7:58pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/7 "2015-05-01T19:58:30Z")
No, for two reasons: 1. I am using the exact same code and data on both machines. 2. I've seen duplicates in the past and I get an error message. Thanks. Blake On Friday, May 1, 2015 at 2:50:57 PM UTC-5, David Pilato wrote: > Any chance you are using the same id multiple times? > > -- > _David Pilato_ - Developer | Evangelist > _[elastic.co](http://elastic.co) [http://elastic.co](http://elastic.co)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr > [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs > [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 1 mai 2015 à 21:25, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a > écrit : > > I changed the code to read: > > var counter = 0; > > exports.addDoc = function (index, type, id, doc, callback) { > if (client !== undefined) { > var json = { > index: index, > type: type, > id: id, > body: doc > }; > if (counter++ % 1000 === 0) { > console.log('Adding document #' + counter); > } > client.create(json, callback); > } else if (callback !== undefined) { > callback('Elasticsearch not connected', undefined); > } > }; > > The last printout reads: Adding document #53001 > > The code that does the error check looks like: > > esUtils.addDoc(esIndex, 'component', compObj.id, doc, function (err, response, status) { > if (err !== undefined || status !== 201 || response.created !== true) { > console.log('Unexpected ES response: ' + status + ' ' + err + response.created); > } > }); > > I never see that message. Finally, after the above I get: > > $ curl -s -XPOST '[http://localhost:9200/components/\_count](http://localhost:9200/components/_count)' > {"count":10500,"\_shards":{"total":5,"successful":5,"failed":0}} > > Thanks for the help! > > On Friday, May 1, 2015 at 1:57:42 PM UTC-5, David Pilato wrote: > > > Could you add a counter in your JS app to make sure you sent all docs? > > > > I suspect something wrong in your index process > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 1 mai 2015 à 20:40, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > The log only contains: > > > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] > > [mmsapp-na-component] [components-1430504530354] creating index, cause > > [api], templates , shards [5]/[1], mappings [index\_name, component] > > > > Each document is being added individually from JavaScript via: > > > > exports.addDoc = function (index, type, id, doc, callback) { > > if (client !== undefined) { > > var json = { > > index: index, > > type: type, > > id: id, > > body: doc > > }; > > client.create(json, callback); > > } else if (callback !== undefined) { > > callback('Elasticsearch not connected', undefined); > > } > > }; > > > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > > > > If you have nothing in logs it could mean that you have an issue with > > > your injector. > > > May be you are using bulk but you don't check the bulk response? > > > > > > David > > > > > > Le 1 mai 2015 à 18:36, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > > > Greetings, > > > > > > I have two similar but unrelated machines. I am adding 50,000+ > > > documents to each. Afterwards, one shows the 50,000+ documents and the > > > other only shows 10,500. The second machine seems to be capping out at > > > 10,500. Why, and how can I correct this? The relevant facts are as > > > follows: > > > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB of > > > RAM and more than sufficient disk space. > > > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is > > > running local to each machine. > > > > > > 3. Both machines are running the exact same program to load up ES. > > > Each has nearly default ES config files (just different names). > > > > > > 4. The program keeps a counter of the number of times documents are > > > added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > > > 5. When I do a the same query on each machine with curl, the good > > > machine shows a max\_score of 8.2, the bad machine shows .499 - remember, > > > same set of documents and same search query. > > > > > > I've spent a day on this, and I am running out of ideas. Any help would > > > sure be appreciated. > > > > > > Blake McBride > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 1, 2015, 8:38pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/8 "2015-05-01T20:38:47Z")
Could you compare disk size (/data dir) for your two elasticsearch instances? Also, could you GIST the result of a simple \_search?pretty on both nodes? -- David Pilato - Developer | Evangelist > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co/)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > Le 1 mai 2015 à 21:58, Blake McBride [blake1024@gmail.com](mailto:blake1024@gmail.com) a écrit : > > No, for two reasons: > > 1. I am using the exact same code and data on both machines. > > 2. I've seen duplicates in the past and I get an error message. > > Thanks. > > Blake > > On Friday, May 1, 2015 at 2:50:57 PM UTC-5, David Pilato wrote: > Any chance you are using the same id multiple times? > > -- > David Pilato - Developer | Evangelist > [elastic.co](http://elastic.co) [http://elastic.co/](http://elastic.co/) > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > Le 1 mai 2015 à 21:25, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a écrit : > > > > I changed the code to read: > > > > var counter = 0; > > > > exports.addDoc = function (index, type, id, doc, callback) { > > if (client !== undefined) { > > var json = { > > index: index, > > type: type, > > id: id, > > body: doc > > }; > > if (counter++ % 1000 === 0) { > > console.log('Adding document #' + counter); > > } > > client.create(json, callback); > > } else if (callback !== undefined) { > > callback('Elasticsearch not connected', undefined); > > } > > }; > > > > The last printout reads: Adding document #53001 > > > > The code that does the error check looks like: > > > > esUtils.addDoc(esIndex, 'component', compObj.id, doc, function (err, response, status) { > > if (err !== undefined || status !== 201 || response.created !== true) { > > console.log('Unexpected ES response: ' + status + ' ' + err + response.created); > > } > > }); > > > > I never see that message. Finally, after the above I get: > > > > $ curl -s -XPOST '[http://localhost:9200/components/\_count](http://localhost:9200/components/_count)' [http://localhost:9200/components/\_count'](http://localhost:9200/components/_count') > > {"count":10500,"\_shards":{"total":5,"successful":5,"failed":0}} > > > > Thanks for the help! > > > > On Friday, May 1, 2015 at 1:57:42 PM UTC-5, David Pilato wrote: > > Could you add a counter in your JS app to make sure you sent all docs? > > > > I suspect something wrong in your index process > > > > -- > > David 😉 > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > Le 1 mai 2015 à 20:40, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \<\>\> a écrit : > > > > > The log only contains: > > > > > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] [mmsapp-na-component] [components-1430504530354] creating index, cause [api], templates , shards [5]/[1], mappings [index\_name, component] > > > > > > Each document is being added individually from JavaScript via: > > > > > > exports.addDoc = function (index, type, id, doc, callback) { > > > if (client !== undefined) { > > > var json = { > > > index: index, > > > type: type, > > > id: id, > > > body: doc > > > }; > > > client.create(json, callback); > > > } else if (callback !== undefined) { > > > callback('Elasticsearch not connected', undefined); > > > } > > > }; > > > > > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > > If you have nothing in logs it could mean that you have an issue with your injector. > > > May be you are using bulk but you don't check the bulk response? > > > > > > David > > > > > > Le 1 mai 2015 à 18:36, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \<\>\> a écrit : > > > > > > > Greetings, > > > > > > > > I have two similar but unrelated machines. I am adding 50,000+ documents to each. Afterwards, one shows the 50,000+ documents and the other only shows 10,500. The second machine seems to be capping out at 10,500. Why, and how can I correct this? The relevant facts are as follows: > > > > > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB of RAM and more than sufficient disk space. > > > > > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is running local to each machine. > > > > > > > > 3. Both machines are running the exact same program to load up ES. Each has nearly default ES config files (just different names). > > > > > > > > 4. The program keeps a counter of the number of times documents are added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > > > > > 5. When I do a the same query on each machine with curl, the good machine shows a max\_score of 8.2, the bad machine shows .499 - remember, same set of documents and same search query. > > > > > > > > I've spent a day on this, and I am running out of ideas. Any help would sure be appreciated. > > > > > > > > Blake McBride > > > > > > > > -- > > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<\>. > > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer). > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<\>. > > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm_medium=email&utm_source=footer). > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm_medium=email&utm_source=footer). > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com) [https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com?utm_medium=email&utm_source=footer). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout) [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2A3198F3-AF63-4D90-8DC4-60C255297AFB%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/2A3198F3-AF63-4D90-8DC4-60C255297AFB%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![blake1024](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@blake1024](https://discuss.elastic.co/u/blake1024)\ **Post date:** [May 1, 2015, 9:41pm UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/9 "2015-05-01T21:41:36Z")
The question of relative size has, I believe, led me to the problem. I create aliases. After the load, I have the alias point to the new index. One of the indexes had a bad document that made the change alias mechanism fail. This means I kept loading the document into an index but the alias was always pointing to an old index. So, on the working system the database size was relatively small - since I got rid of the old indexes. The bad machine was taking up a lot of space because the old indexes were never deleted because the alias didn't point to them. Thanks a lot for the help!! Blake On Friday, May 1, 2015 at 3:39:05 PM UTC-5, David Pilato wrote: > Could you compare disk size (/data dir) for your two elasticsearch > instances? > Also, could you GIST the result of a simple \_search?pretty on both nodes? > > -- > _David Pilato_ - Developer | Evangelist > _[elastic.co](http://elastic.co) [http://elastic.co](http://elastic.co)_ > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr > [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs > [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > Le 1 mai 2015 à 21:58, Blake McBride \<[blak...@gmail.com](mailto:blak...@gmail.com) \\> a > écrit : > > No, for two reasons: > > 1. I am using the exact same code and data on both machines. > > 2. I've seen duplicates in the past and I get an error message. > > Thanks. > > Blake > > On Friday, May 1, 2015 at 2:50:57 PM UTC-5, David Pilato wrote: > > > Any chance you are using the same id multiple times? > > > > -- > > _David Pilato_ - Developer | Evangelist > > _[elastic.co](http://elastic.co) [http://elastic.co/](http://elastic.co/)_ > > @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr > > [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs > > [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > > > > Le 1 mai 2015 à 21:25, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > I changed the code to read: > > > > var counter = 0; > > > > exports.addDoc = function (index, type, id, doc, callback) { > > if (client !== undefined) { > > var json = { > > index: index, > > type: type, > > id: id, > > body: doc > > }; > > if (counter++ % 1000 === 0) { > > console.log('Adding document #' + counter); > > } > > client.create(json, callback); > > } else if (callback !== undefined) { > > callback('Elasticsearch not connected', undefined); > > } > > }; > > > > The last printout reads: Adding document #53001 > > > > The code that does the error check looks like: > > > > esUtils.addDoc(esIndex, 'component', compObj.id, doc, function (err, response, status) { > > if (err !== undefined || status !== 201 || response.created !== true) { > > console.log('Unexpected ES response: ' + status + ' ' + err + response.created); > > } > > }); > > > > I never see that message. Finally, after the above I get: > > > > $ curl -s -XPOST '[http://localhost:9200/components/\_count](http://localhost:9200/components/_count)' > > {"count":10500,"\_shards":{"total":5,"successful":5,"failed":0}} > > > > Thanks for the help! > > > > On Friday, May 1, 2015 at 1:57:42 PM UTC-5, David Pilato wrote: > > > > > Could you add a counter in your JS app to make sure you sent all docs? > > > > > > I suspect something wrong in your index process > > > > > > -- > > > David 😉 > > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > > > > Le 1 mai 2015 à 20:40, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > > > The log only contains: > > > > > > [2015-05-01 18:22:10,398][INFO][cluster.metadata] > > > [mmsapp-na-component] [components-1430504530354] creating index, cause > > > [api], templates , shards [5]/[1], mappings [index\_name, component] > > > > > > Each document is being added individually from JavaScript via: > > > > > > exports.addDoc = function (index, type, id, doc, callback) { > > > if (client !== undefined) { > > > var json = { > > > index: index, > > > type: type, > > > id: id, > > > body: doc > > > }; > > > client.create(json, callback); > > > } else if (callback !== undefined) { > > > callback('Elasticsearch not connected', undefined); > > > } > > > }; > > > > > > On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: > > > > > > > If you have nothing in logs it could mean that you have an issue with > > > > your injector. > > > > May be you are using bulk but you don't check the bulk response? > > > > > > > > David > > > > > > > > Le 1 mai 2015 à 18:36, Blake McBride [blak...@gmail.com](mailto:blak...@gmail.com) a écrit : > > > > > > > > Greetings, > > > > > > > > I have two similar but unrelated machines. I am adding 50,000+ > > > > documents to each. Afterwards, one shows the 50,000+ documents and the > > > > other only shows 10,500. The second machine seems to be capping out at > > > > 10,500. Why, and how can I correct this? The relevant facts are as > > > > follows: > > > > > > > > 1. Both machines are current 64 bit Linux machines with at least 8GB > > > > of RAM and more than sufficient disk space. > > > > > > > > 2. Both have current 64 bit Java 7 and elasticsearch 1.5.1. ES is > > > > running local to each machine. > > > > > > > > 3. Both machines are running the exact same program to load up ES. > > > > Each has nearly default ES config files (just different names). > > > > > > > > 4. The program keeps a counter of the number of times documents are > > > > added to ES, and the return codes of each add is checked. Both are 50,000+. > > > > > > > > 5. When I do a the same query on each machine with curl, the good > > > > machine shows a max\_score of 8.2, the bad machine shows .499 - remember, > > > > same set of documents and same search query. > > > > > > > > I've spent a day on this, and I am running out of ideas. Any help > > > > would sure be appreciated. > > > > > > > > Blake McBride > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c44faf88-1b67-4843-a6ed-52d31c055716%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/cc41f76d-9716-4f06-9f3a-ec18f0e07bf0%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f4c6434c-2dc2-49c3-a331-e5f88204e239%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/b4a6a6df-73aa-44ce-bab1-20db2ed1e3c7%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2c5ae437-5a65-4528-b33c-edc133e542c1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2c5ae437-5a65-4528-b33c-edc133e542c1%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:16am UTC](https://discuss.elastic.co/t/max-documents-10-500/23514/10 "2017-07-06T00:16:12Z")