# Query Advice Needed **URL:** \ **Category:** Elasticsearch\ **Created:** [May 3, 2011, 9:13pm UTC](https://discuss.elastic.co/t/query-advice-needed/4336 "2011-05-03T21:13:30Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![frazer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frazer/32/2752_2.png) [@frazer](https://discuss.elastic.co/u/frazer)\ **Post date:** [May 3, 2011, 9:13pm UTC](https://discuss.elastic.co/t/query-advice-needed/4336/1 "2011-05-03T21:13:30Z")
I have a situation where I need to query an index and return a 1000 documents. This query will be frequently run by many users. Am I best to a) get the 1000 documents in 1 query b) break the query down into smaller ones, manual paging c) use the scan search type and scroll for 1k documents (sort is not an issue) Just wondering out of these options which would be most efficient. Thanks in advance Frazer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 3, 2011, 9:17pm UTC](https://discuss.elastic.co/t/query-advice-needed/4336/2 "2011-05-03T21:17:53Z")
If those are small documents, then just getting all of them is fine. I plan to add a "no\_sort" search type (or similar) to make it even less taxing. Make sure you use query\_and\_fetch, which will make it even faster (I assume you want all docs that match that query). On Wednesday, May 4, 2011 at 12:13 AM, frazer wrote: > I have a situation where I need to query an index and return a 1000 > documents. This query will be frequently run by many users. > > Am I best to > > a) get the 1000 documents in 1 query > b) break the query down into smaller ones, manual paging > c) use the scan search type and scroll for 1k documents (sort is not > an issue) > > Just wondering out of these options which would be most efficient. > > Thanks in advance > > Frazer ---
**Author:** ![frazer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frazer/32/2752_2.png) [@frazer](https://discuss.elastic.co/u/frazer)\ **Post date:** [May 4, 2011, 12:37am UTC](https://discuss.elastic.co/t/query-advice-needed/4336/3 "2011-05-04T00:37:07Z")
Great, thanks Shay. The documents have about 40 fields with really only 1 of those fields potentially containing any sizeable text. Would that qualify as a small document? Also, I only really need ids, is there much of a performance gain doing an id only query? Yes, a no sort search type would be perfect for my scenario. Thanks again On May 3, 5:17 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > If those are small documents, then just getting all of them is fine. I plan to add a "no\_sort" search type (or similar) to make it even less taxing. Make sure you use query\_and\_fetch, which will make it even faster (I assume you want all docs that match that query). > > On Wednesday, May 4, 2011 at 12:13 AM, frazer wrote: > > > I have a situation where I need to query an index and return a 1000 > > documents. This query will be frequently run by many users. > > > Am I best to > > > a) get the 1000 documents in 1 query > > b) break the query down into smaller ones, manual paging > > c) use the scan search type and scroll for 1k documents (sort is not > > an issue) > > > Just wondering out of these options which would be most efficient. > > > Thanks in advance > > > Frazer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 4, 2011, 6:44am UTC](https://discuss.elastic.co/t/query-advice-needed/4336/4 "2011-05-04T06:44:15Z")
If you just want to get the ids, then providing an empty fields array in the search request will return it, so the size of the document is not a problem. And, in this case, getting al the results back is certainly something that you should do. On Wednesday, May 4, 2011 at 3:37 AM, frazer wrote: > Great, thanks Shay. > > The documents have about 40 fields with really only 1 of those fields > potentially containing any sizeable text. Would that qualify as a > small document? > > Also, I only really need ids, is there much of a performance gain > doing an id only query? > > Yes, a no sort search type would be perfect for my scenario. > > Thanks again > > On May 3, 5:17 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > If those are small documents, then just getting all of them is fine. I plan to add a "no\_sort" search type (or similar) to make it even less taxing. Make sure you use query\_and\_fetch, which will make it even faster (I assume you want all docs that match that query). > > > > On Wednesday, May 4, 2011 at 12:13 AM, frazer wrote: > > > > > I have a situation where I need to query an index and return a 1000 > > > documents. This query will be frequently run by many users. > > > > > Am I best to > > > > > a) get the 1000 documents in 1 query > > > b) break the query down into smaller ones, manual paging > > > c) use the scan search type and scroll for 1k documents (sort is not > > > an issue) > > > > > Just wondering out of these options which would be most efficient. > > > > > Thanks in advance > > > > > Frazer ---
**Author:** ![frazer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/frazer/32/2752_2.png) [@frazer](https://discuss.elastic.co/u/frazer)\ **Post date:** [May 4, 2011, 11:50am UTC](https://discuss.elastic.co/t/query-advice-needed/4336/5 "2011-05-04T11:50:16Z")
Thanks Shay, so It sounds like its more efficient for me just to get back the ids. Thanks for the advice On May 4, 2:44 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > If you just want to get the ids, then providing an empty fields array in the search request will return it, so the size of the document is not a problem. And, in this case, getting al the results back is certainly something that you should do. > > On Wednesday, May 4, 2011 at 3:37 AM, frazer wrote: > > > Great, thanks Shay. > > > The documents have about 40 fields with really only 1 of those fields > > potentially containing any sizeable text. Would that qualify as a > > small document? > > > Also, I only really need ids, is there much of a performance gain > > doing an id only query? > > > Yes, a no sort search type would be perfect for my scenario. > > > Thanks again > > > On May 3, 5:17 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > If those are small documents, then just getting all of them is fine. I plan to add a "no\_sort" search type (or similar) to make it even less taxing. Make sure you use query\_and\_fetch, which will make it even faster (I assume you want all docs that match that query). > > > > On Wednesday, May 4, 2011 at 12:13 AM, frazer wrote: > > > > > > > I have a situation where I need to query an index and return a 1000 > > > > documents. This query will be frequently run by many users. > > > > > Am I best to > > > > > a) get the 1000 documents in 1 query > > > > b) break the query down into smaller ones, manual paging > > > > c) use the scan search type and scroll for 1k documents (sort is not > > > > an issue) > > > > > Just wondering out of these options which would be most efficient. > > > > > Thanks in advance > > > > > Frazer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 4, 2011, 1:42pm UTC](https://discuss.elastic.co/t/query-advice-needed/4336/6 "2011-05-04T13:42:34Z")
Getting back just hte ids means much less data to get back. Its up to what you really need, getting the \_source from ES compared to the DB will be faster (because its "already" there). On Wednesday, May 4, 2011 at 2:50 PM, frazer wrote: > Thanks Shay, so It sounds like its more efficient for me just to get > back the ids. > > Thanks for the advice > > On May 4, 2:44 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > If you just want to get the ids, then providing an empty fields array in the search request will return it, so the size of the document is not a problem. And, in this case, getting al the results back is certainly something that you should do. > > > > On Wednesday, May 4, 2011 at 3:37 AM, frazer wrote: > > > > > Great, thanks Shay. > > > > > The documents have about 40 fields with really only 1 of those fields > > > potentially containing any sizeable text. Would that qualify as a > > > small document? > > > > > Also, I only really need ids, is there much of a performance gain > > > doing an id only query? > > > > > Yes, a no sort search type would be perfect for my scenario. > > > > > Thanks again > > > > > On May 3, 5:17 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > > > If those are small documents, then just getting all of them is fine. I plan to add a "no\_sort" search type (or similar) to make it even less taxing. Make sure you use query\_and\_fetch, which will make it even faster (I assume you want all docs that match that query). > > > > > > On Wednesday, May 4, 2011 at 12:13 AM, frazer wrote: > > > > > > > > > I have a situation where I need to query an index and return a 1000 > > > > > documents. This query will be frequently run by many users. > > > > > > > Am I best to > > > > > > > a) get the 1000 documents in 1 query > > > > > b) break the query down into smaller ones, manual paging > > > > > c) use the scan search type and scroll for 1k documents (sort is not > > > > > an issue) > > > > > > > Just wondering out of these options which would be most efficient. > > > > > > > Thanks in advance > > > > > > > Frazer ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:06am UTC](https://discuss.elastic.co/t/query-advice-needed/4336/7 "2017-07-06T04:06:58Z")