# List all entries matching a query **URL:** \ **Category:** Elasticsearch\ **Created:** [September 14, 2010, 6:38am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332 "2010-09-14T06:38:50Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![sezgin\_kucukkaraasla](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@sezgin\_kucukkaraasla](https://discuss.elastic.co/u/sezgin_kucukkaraasla)\ **Post date:** [September 14, 2010, 6:38am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/1 "2010-09-14T06:38:50Z")
Is there a way to list all entries matching a query, without providing the size parameter, because we don't know the actual size beforehand ? Sezgin Kucukkaraaslan [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [September 14, 2010, 6:46am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/2 "2010-09-14T06:46:56Z")
I don't think there is but you can learn the needed size using the count query: [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/count/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/count/) it should be efficient. Rgds, Lukas 2010/9/14 sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > Is there a way to list all entries matching a query, without providing the > size parameter, because we don't know the actual size beforehand ? > > Sezgin Kucukkaraaslan > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Andrew\_Harvey](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@Andrew\_Harvey](https://discuss.elastic.co/u/Andrew_Harvey)\ **Post date:** [September 14, 2010, 6:48am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/3 "2010-09-14T06:48:13Z")
Set size to a Really Big Number™ A. On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > Is there a way to list all entries matching a query, without providing the size parameter, because we don't know the actual size beforehand ? > > Sezgin Kucukkaraaslan > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [September 14, 2010, 6:53am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/4 "2010-09-14T06:53:48Z")
Then I guess you would want to check [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as well. Generally I am not sure Elasticsearch is designed to return a huge amount of documents in a single response. On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > Set size to a Really Big Number™ > > A. > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > Is there a way to list all entries matching a query, without providing the > size parameter, because we don't know the actual size beforehand ? > > Sezgin Kucukkaraaslan > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Andrew\_Harvey](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@Andrew\_Harvey](https://discuss.elastic.co/u/Andrew_Harvey)\ **Post date:** [September 14, 2010, 6:56am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/5 "2010-09-14T06:56:39Z")
I've found scroll to be patchy at best. There is an open bug for it. The Really Big Number™ approach has its flaws, like the network load and the possibility that you could knock over your client or your server if the response generated is larger than either can handle. However, if your response size is always going to be \< 1000 documents and they're of a reasonable size, you can probably get away with it. A. On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > Then I guess you would want to check [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as well. Generally I am not sure Elasticsearch is designed to return a huge amount of documents in a single response. > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > Set size to a Really Big Number™ > > A. > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > > Is there a way to list all entries matching a query, without providing the size parameter, because we don't know the actual size beforehand ? > > > > Sezgin Kucukkaraaslan > > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![sezgin\_kucukkaraasla](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@sezgin\_kucukkaraasla](https://discuss.elastic.co/u/sezgin_kucukkaraasla)\ **Post date:** [September 14, 2010, 7:28am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/6 "2010-09-14T07:28:38Z")
The Really Big Number approach was the first thing that came to my mind. But I didn't want to provide 1000000 as size for 200 entries fearing that some memory space might have been reserved beforehand, or something else hurting the performance. If this scenario is ok with the performance, I can use it. On Tue, Sep 14, 2010 at 9:56 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > I've found scroll to be patchy at best. There is an open bug for it. > > The Really Big Number™ approach has its flaws, like the network load and > the possibility that you could knock over your client or your server if the > response generated is larger than either can handle. However, if your > response size is always going to be \< 1000 documents and they're of a > reasonable size, you can probably get away with it. > > A. > > On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > > Then I guess you would want to check > [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/search/scroll/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as > well. Generally I am not sure Elasticsearch is designed to return a huge > amount of documents in a single response. > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > > > Set size to a Really Big Number™ > > > > A. > > > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > > > Is there a way to list all entries matching a query, without providing the > > size parameter, because we don't know the actual size beforehand ? > > > > Sezgin Kucukkaraaslan > > [www.ifountain.com](http://www.ifountain.com) ---
**Author:** ![Paul\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_smith/32/1323_2.png) [@Paul\_Smith](https://discuss.elastic.co/u/Paul_Smith)\ **Post date:** [September 14, 2010, 7:49am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/7 "2010-09-14T07:49:51Z")
You definitely don't want to use just any big number because unless ES does something extra fancy (maybe!) Lucene will allocate a PriorityQueue with an array the size of your number of results. This may 'work' but will not survive real world use I suspect. The count API is what you want but it would be nice if ES could be given a value that means 'all' results and it do the count in in the one RPC call saving the round trip. Paul On Tuesday, September 14, 2010, sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) wrote: > The Really Big Number approach was the first thing that came to my mind. But I didn't want to provide 1000000 as size for 200 entries fearing that some memory space might have been reserved beforehand, or something else hurting the performance. If this scenario is ok with the performance, I can use it. > > On Tue, Sep 14, 2010 at 9:56 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > > I've found scroll to be patchy at best. There is an open bug for it. > The Really Big Number™ approach has its flaws, like the network load and the possibility that you could knock over your client or your server if the response generated is larger than either can handle. However, if your response size is always going to be \< 1000 documents and they're of a reasonable size, you can probably get away with it. > > A. > On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > Then I guess you would want to check [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as well. Generally I am not sure Elasticsearch is designed to return a huge amount of documents in a single response. > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) wrote: > > Set size to a Really Big Number™ > > A. > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > Is there a way to list all entries matching a query, without providing the size parameter, because we don't know the actual size beforehand ? > > Sezgin Kucukkaraaslan > [www.ifountain.com](http://www.ifountain.com) [http://www.ifountain.com/](http://www.ifountain.com/) ---
**Author:** ![sezgin\_kucukkaraasla](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@sezgin\_kucukkaraasla](https://discuss.elastic.co/u/sezgin_kucukkaraasla)\ **Post date:** [September 14, 2010, 8:38am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/8 "2010-09-14T08:38:53Z")
Thanks for the notice, Yes, If ES can provide something like "all", that would be great. Sezgin Kucukkaraaslan On Tue, Sep 14, 2010 at 10:49 AM, Paul Smith [tallpsmith@gmail.com](mailto:tallpsmith@gmail.com) wrote: > You definitely don't want to use just any big number because unless ES > does something extra fancy (maybe!) Lucene will allocate a > PriorityQueue with an array the size of your number of results. This > may 'work' but will not survive real world use I suspect. > > The count API is what you want but it would be nice if ES could be > given a value that means 'all' results and it do the count in in the > one RPC call saving the round trip. > > Paul > > On Tuesday, September 14, 2010, sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > wrote: > > > The Really Big Number approach was the first thing that came to my mind. > > But I didn't want to provide 1000000 as size for 200 entries fearing that > > some memory space might have been reserved beforehand, or something else > > hurting the performance. If this scenario is ok with the performance, I can > > use it. > > > > On Tue, Sep 14, 2010 at 9:56 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > wrote: > > > > I've found scroll to be patchy at best. There is an open bug for it. > > The Really Big Number™ approach has its flaws, like the network load and > > the possibility that you could knock over your client or your server if the > > response generated is larger than either can handle. However, if your > > response size is always going to be \< 1000 documents and they're of a > > reasonable size, you can probably get away with it. > > > > A. > > On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > > Then I guess you would want to check > > [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as > > well. Generally I am not sure Elasticsearch is designed to return a huge > > amount of documents in a single response. > > > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > wrote: > > > > Set size to a Really Big Number™ > > > > A. > > > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > > > Is there a way to list all entries matching a query, without providing > > the size parameter, because we don't know the actual size beforehand ? > > > > Sezgin Kucukkaraaslan > > [www.ifountain.com](http://www.ifountain.com) [http://www.ifountain.com/](http://www.ifountain.com/) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 14, 2010, 1:39pm UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/9 "2010-09-14T13:39:52Z")
This can be done (though will be a bit slower). Open an issue for this. 2010/9/14 sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > Thanks for the notice, > Yes, If ES can provide something like "all", that would be great. > > Sezgin Kucukkaraaslan > > On Tue, Sep 14, 2010 at 10:49 AM, Paul Smith [tallpsmith@gmail.com](mailto:tallpsmith@gmail.com) wrote: > > > You definitely don't want to use just any big number because unless ES > > does something extra fancy (maybe!) Lucene will allocate a > > PriorityQueue with an array the size of your number of results. This > > may 'work' but will not survive real world use I suspect. > > > > The count API is what you want but it would be nice if ES could be > > given a value that means 'all' results and it do the count in in the > > one RPC call saving the round trip. > > > > Paul > > > > On Tuesday, September 14, 2010, sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > > wrote: > > > > > The Really Big Number approach was the first thing that came to my mind. > > > But I didn't want to provide 1000000 as size for 200 entries fearing that > > > some memory space might have been reserved beforehand, or something else > > > hurting the performance. If this scenario is ok with the performance, I can > > > use it. > > > > > > On Tue, Sep 14, 2010 at 9:56 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > > wrote: > > > > > > I've found scroll to be patchy at best. There is an open bug for it. > > > The Really Big Number™ approach has its flaws, like the network load and > > > the possibility that you could knock over your client or your server if the > > > response generated is larger than either can handle. However, if your > > > response size is always going to be \< 1000 documents and they're of a > > > reasonable size, you can probably get away with it. > > > > > > A. > > > On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > > > Then I guess you would want to check > > > [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as > > > well. Generally I am not sure Elasticsearch is designed to return a huge > > > amount of documents in a single response. > > > > > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > > wrote: > > > > > > Set size to a Really Big Number™ > > > > > > A. > > > > > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > > > > > Is there a way to list all entries matching a query, without providing > > > the size parameter, because we don't know the actual size beforehand ? > > > > > > Sezgin Kucukkaraaslan > > > [www.ifountain.com](http://www.ifountain.com) [http://www.ifountain.com/](http://www.ifountain.com/) ---
**Author:** ![sezgin\_kucukkaraasla](https://avatars.discourse-cdn.com/v4/letter/s/9de0a6/32.png) [@sezgin\_kucukkaraasla](https://discuss.elastic.co/u/sezgin_kucukkaraasla)\ **Post date:** [September 15, 2010, 10:42am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/10 "2010-09-15T10:42:47Z")
Thanks... On Tue, Sep 14, 2010 at 4:39 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > This can be done (though will be a bit slower). Open an issue for this. > > 2010/9/14 sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > > Thanks for the notice, > > > Yes, If ES can provide something like "all", that would be great. > > > > Sezgin Kucukkaraaslan > > > > On Tue, Sep 14, 2010 at 10:49 AM, Paul Smith [tallpsmith@gmail.com](mailto:tallpsmith@gmail.com)wrote: > > > > > You definitely don't want to use just any big number because unless ES > > > does something extra fancy (maybe!) Lucene will allocate a > > > PriorityQueue with an array the size of your number of results. This > > > may 'work' but will not survive real world use I suspect. > > > > > > The count API is what you want but it would be nice if ES could be > > > given a value that means 'all' results and it do the count in in the > > > one RPC call saving the round trip. > > > > > > Paul > > > > > > On Tuesday, September 14, 2010, sezgin küçükkaraaslan [sezo104@gmail.com](mailto:sezo104@gmail.com) > > > wrote: > > > > > > > The Really Big Number approach was the first thing that came to my > > > > mind. But I didn't want to provide 1000000 as size for 200 entries fearing > > > > that some memory space might have been reserved beforehand, or something > > > > else hurting the performance. If this scenario is ok with the performance, I > > > > can use it. > > > > > > > > On Tue, Sep 14, 2010 at 9:56 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > > > wrote: > > > > > > > > I've found scroll to be patchy at best. There is an open bug for it. > > > > The Really Big Number™ approach has its flaws, like the network load > > > > and the possibility that you could knock over your client or your server if > > > > the response generated is larger than either can handle. However, if your > > > > response size is always going to be \< 1000 documents and they're of a > > > > reasonable size, you can probably get away with it. > > > > > > > > A. > > > > On 14/09/2010, at 4:53 PM, Lukáš Vlček wrote: > > > > Then I guess you would want to check > > > > [Request body search | Elasticsearch Guide [8.11] | Elastic](http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/scroll/) as > > > > well. Generally I am not sure Elasticsearch is designed to return a huge > > > > amount of documents in a single response. > > > > > > > > On Tue, Sep 14, 2010 at 8:48 AM, Andrew Harvey [synapsys@gmail.com](mailto:synapsys@gmail.com) > > > > wrote: > > > > > > > > Set size to a Really Big Number™ > > > > > > > > A. > > > > > > > > On 14/09/2010, at 4:38 PM, sezgin küçükkaraaslan wrote: > > > > > > > > Is there a way to list all entries matching a query, without providing > > > > the size parameter, because we don't know the actual size beforehand ? > > > > > > > > Sezgin Kucukkaraaslan > > > > [www.ifountain.com](http://www.ifountain.com) [http://www.ifountain.com/](http://www.ifountain.com/) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:19am UTC](https://discuss.elastic.co/t/list-all-entries-matching-a-query/3332/11 "2017-07-06T04:19:19Z")