# Non-Paged Query And Performance **URL:** \ **Category:** Elasticsearch\ **Created:** [March 7, 2011, 10:18pm UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051 "2011-03-07T22:18:51Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![timscott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timscott/32/3220_2.png) [@timscott](https://discuss.elastic.co/u/timscott)\ **Post date:** [March 7, 2011, 10:18pm UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051/1 "2011-03-07T22:18:51Z")
I understand that the default page size is 10 hits, which can be changed via config or in a query by specifying "from" and "size". I have a scenario where I want all hits. I can be reasonably confident that the results set will not be too large. I tried setting size to 0, -1 and max-int (in C#). The first two returned no results. The latter seems to work, functionally speaking, but it seems there is a price to pay. My queries are timing out even though the number of hits would be not too large (maybe 100-200). I could set a high number but much lower than max int. But I wonder, does performance always deteriorate with page size, even if there are few hits? Is there a recommended way to approach this scenario? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 8, 2011, 6:45am UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051/2 "2011-03-08T06:45:30Z")
Hey, If you ask for N hits, a priority queue with that size (N) is built by Lucene in order to have a data structure that keeps the hits ordered. So, if you ask for something like MAX\_INT-1, it is _incredibly_ wasteful. So, you should ask for something like 500 hits (if you know you are going to have less). The new "scan" search type in master can help in certain scenarios (it does no sorting) as it bypass the need to sort. -shay.banon On Tuesday, March 8, 2011 at 12:18 AM, Tim Scott wrote: I understand that the default page size is 10 hits, which can be > changed via config or in a query by specifying "from" and "size". I > have a scenario where I want all hits. I can be reasonably confident > that the results set will not be too large. I tried setting size to > 0, -1 and max-int (in C#). The first two returned no results. The > latter seems to work, functionally speaking, but it seems there is a > price to pay. My queries are timing out even though the number of > hits would be not too large (maybe 100-200). > > I could set a high number but much lower than max int. But I wonder, > does performance always deteriorate with page size, even if there are > few hits? Is there a recommended way to approach this scenario? ---
**Author:** ![timscott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timscott/32/3220_2.png) [@timscott](https://discuss.elastic.co/u/timscott)\ **Post date:** [March 8, 2011, 6:01pm UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051/3 "2011-03-08T18:01:35Z")
You know what I really need...I only need the IDs of the documents that match the query. But I need all of them. Any really efficient way to do that? Tim On Mar 8, 12:45 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hey, > > If you ask for N hits, a priority queue with that size (N) is built by Lucene in order to have a data structure that keeps the hits ordered. So, if you ask for something like MAX\_INT-1, it is _incredibly_ wasteful. So, you should ask for something like 500 hits (if you know you are going to have less). > > The new "scan" search type in master can help in certain scenarios (it does no sorting) as it bypass the need to sort. > > -shay.banonOn Tuesday, March 8, 2011 at 12:18 AM, Tim Scott wrote: > > I understand that the default page size is 10 hits, which can be > > > changed via config or in a query by specifying "from" and "size". I > > have a scenario where I want all hits. I can be reasonably confident > > that the results set will not be too large. I tried setting size to > > 0, -1 and max-int (in C#). The first two returned no results. The > > latter seems to work, functionally speaking, but it seems there is a > > price to pay. My queries are timing out even though the number of > > hits would be not too large (maybe 100-200). > > > I could set a high number but much lower than max int. But I wonder, > > does performance always deteriorate with page size, even if there are > > few hits? Is there a recommended way to approach this scenario? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 8, 2011, 6:25pm UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051/4 "2011-03-08T18:25:31Z")
Getting only the ids matching is simple, by specifying an empty array of fields. Getting all of them require you to set a size (and make sure you use query\_and\_fetch search type). It can be optimized, I guess, by introducing a new search\_type that will return the full set of results, without any ordering. Its similar to the new scan type, though without the overhead of setting up scanning. -shay.banon On Tuesday, March 8, 2011 at 8:01 PM, Tim Scott wrote: > You know what I really need...I only need the IDs of the documents > that match the query. But I need all of them. Any really efficient > way to do that? > > Tim > > On Mar 8, 12:45 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Hey, > > > > If you ask for N hits, a priority queue with that size (N) is built by Lucene in order to have a data structure that keeps the hits ordered. So, if you ask for something like MAX\_INT-1, it is _incredibly_ wasteful. So, you should ask for something like 500 hits (if you know you are going to have less). > > > > The new "scan" search type in master can help in certain scenarios (it does no sorting) as it bypass the need to sort. > > > > -shay.banonOn Tuesday, March 8, 2011 at 12:18 AM, Tim Scott wrote: > > > > I understand that the default page size is 10 hits, which can be > > > > > changed via config or in a query by specifying "from" and "size". I > > > have a scenario where I want all hits. I can be reasonably confident > > > that the results set will not be too large. I tried setting size to > > > 0, -1 and max-int (in C#). The first two returned no results. The > > > latter seems to work, functionally speaking, but it seems there is a > > > price to pay. My queries are timing out even though the number of > > > hits would be not too large (maybe 100-200). > > > > > I could set a high number but much lower than max int. But I wonder, > > > does performance always deteriorate with page size, even if there are > > > few hits? Is there a recommended way to approach this scenario? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:10am UTC](https://discuss.elastic.co/t/non-paged-query-and-performance/4051/5 "2017-07-06T04:10:31Z")
Closed on July 6, 2017, 4:10am UTC