# Pagination using search type Query\_and\_fetch

**URL:** <https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023>\
**Category:** Elasticsearch\
**Created:** [February 7, 2015, 2:58am UTC](https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023 "2015-02-07T02:58:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hardik\_Dobariya](https://avatars.discourse-cdn.com/v4/letter/h/dbc845/32.png) [@Hardik\_Dobariya](https://discuss.elastic.co/u/Hardik_Dobariya)\
**Post date:** [February 7, 2015, 2:58am UTC](https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023/1 "2015-02-07T02:58:45Z")

</div>

According to Elastic search  
[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch) if  
working with Query\_and\_fetch will return size times number of shards  
hits.Following this rule I implemented below query using Nest having 5  
shards

int from = 0;  
int size = 1;

```
     while (true)
    {
        var result = client.Search<object>(x => x
       .SearchType(SearchType.QueryAndFetch)
       .Size(size)
       .From(from * size)
       .Query(qr => qr.MatchAll())
       );

        if (result.Hits.Count() == 0)
        {
            break;
        }
       from++;
    }

```

If I have total 17 records in index then according to the rule above query  
should execute 4 times returning 5,5,5,2 = 17 total records but this is not  
the case in the above query.  
Above query executes 4 times returning 5,3,3,1 = 12 records missing 5  
records.  
Am I doing anything wrong or missing anything?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\
**Post date:** [February 8, 2015, 7:03pm UTC](https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023/2 "2015-02-08T19:03:49Z")

</div>

I think this is a bug indeed. The QUERY\_AND\_FETCH search mode has been  
designed as an optimization of QUERY\_THE\_FETCH for the single shard case  
(which is typically the case when you use routing) rather than a search  
mode on its own.

As a side note, using QUERY\_AND\_FETCH when paginating on several shards  
would be very bad as it would be much more I/O and memory intensive than  
QUERY\_THEN\_FETCH. So while we could fix it to return correct results when  
paginating, maybe the right fix would rather be to fix documentation to  
document this search type as an internal optinmization for the single-shard  
case which should not be used explicitely.

On Sat, Feb 7, 2015 at 3:58 AM, Hardik Dobariya \<  
[hardikdobariya1132@gmail.com](mailto:hardikdobariya1132@gmail.com)\> wrote:

> According to Elastic search  
> [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch) if  
> working with Query\_and\_fetch will return size times number of shards  
> hits.Following this rule I implemented below query using Nest having 5  
> shards
> 
> int from = 0;  
> int size = 1;
> 
> ```
> while (true)
> {
> var result = client.Search<object>(x => x
> .SearchType(SearchType.QueryAndFetch)
> .Size(size)
> .From(from * size)
> .Query(qr => qr.MatchAll())
> );
> 
> if (result.Hits.Count() == 0)
> {
> break;
> }
> from++;
> }
> 
> ```
> 
> If I have total 17 records in index then according to the rule above query  
> should execute 4 times returning 5,5,5,2 = 17 total records but this is not  
> the case in the above query.  
> Above query executes 4 times returning 5,3,3,1 = 12 records missing 5  
> records.  
> Am I doing anything wrong or missing anything?
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
Adrien Grand

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j66-8r9dd6HXEPm7rcpARVutbWZOKAMUWN0ng5y6Qq3Qw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j66-8r9dd6HXEPm7rcpARVutbWZOKAMUWN0ng5y6Qq3Qw%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Hardik\_Dobariya](https://avatars.discourse-cdn.com/v4/letter/h/dbc845/32.png) [@Hardik\_Dobariya](https://discuss.elastic.co/u/Hardik_Dobariya)\
**Post date:** [February 11, 2015, 5:04am UTC](https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023/3 "2015-02-11T05:04:06Z")

</div>

Thanks Adrien for quick reply,

So I cannot use QUERY\_AND\_FETCH because this would be very expensive  
regarding high I/O and memory intensive.  
The following is the reason I want to use QUERY\_AND\_FETCH because  
If I use QUERY\_THEN\_FETCH then according to Elastic search QUERY\_AND\_FETCH  
is very fast because it does not sort the result on score  
and QUERY\_THEN\_FETCH first fetch result from all shards ,perform sorting  
on score then returns the result.  
I do not want sorting on the result so I want to use QUERY\_AND\_FETCH.  
Is there any other alternative for the same?  
How about using scantype=scan?will it be a good practice to use it or this  
will would also be expensive regarding high I/O and memory intensive

On Monday, February 9, 2015 at 12:33:56 AM UTC+5:30, Adrien Grand wrote:

> I think this is a bug indeed. The QUERY\_AND\_FETCH search mode has been  
> designed as an optimization of QUERY\_THE\_FETCH for the single shard case  
> (which is typically the case when you use routing) rather than a search  
> mode on its own.
> 
> As a side note, using QUERY\_AND\_FETCH when paginating on several shards  
> would be very bad as it would be much more I/O and memory intensive than  
> QUERY\_THEN\_FETCH. So while we could fix it to return correct results when  
> paginating, maybe the right fix would rather be to fix documentation to  
> document this search type as an internal optinmization for the single-shard  
> case which should not be used explicitely.
> 
> On Sat, Feb 7, 2015 at 3:58 AM, Hardik Dobariya \<[hardikdob...@gmail.com](mailto:hardikdob...@gmail.com)  
> \<javascript:\>\> wrote:
> 
> > According to Elastic search  
> > [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#query-and-fetch) if  
> > working with Query\_and\_fetch will return size times number of shards  
> > hits.Following this rule I implemented below query using Nest having 5  
> > shards
> > 
> > int from = 0;  
> > int size = 1;
> > 
> > ```
> > while (true)
> > {
> > var result = client.Search<object>(x => x
> > .SearchType(SearchType.QueryAndFetch)
> > .Size(size)
> > .From(from * size)
> > .Query(qr => qr.MatchAll())
> > );
> > 
> > if (result.Hits.Count() == 0)
> > {
> > break;
> > }
> > from++;
> > }
> > 
> > ```
> > 
> > If I have total 17 records in index then according to the rule above  
> > query should execute 4 times returning 5,5,5,2 = 17 total records but this  
> > is not the case in the above query.  
> > Above query executes 4 times returning 5,3,3,1 = 12 records missing 5  
> > records.  
> > Am I doing anything wrong or missing anything?
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/311c56ad-c3b9-4aa7-b268-9c3c92e21e55%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> Adrien Grand

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b099a9c1-e870-4e50-8515-5265b035ecd2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b099a9c1-e870-4e50-8515-5265b035ecd2%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:33am UTC](https://discuss.elastic.co/t/pagination-using-search-type-query-and-fetch/22023/4 "2017-07-06T00:33:44Z")

</div>