# Search\_after With Collapse Pagination Alternative

**URL:** <https://discuss.elastic.co/t/search-after-with-collapse-pagination-alternative/260451>\
**Category:** Elasticsearch\
**Created:** [January 7, 2021, 2:33pm UTC](https://discuss.elastic.co/t/search-after-with-collapse-pagination-alternative/260451 "2021-01-07T14:33:37Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MilanGatyas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/milangatyas/32/81894_2.png) [@MilanGatyas](https://discuss.elastic.co/u/MilanGatyas)\
**Post date:** [January 7, 2021, 2:33pm UTC](https://discuss.elastic.co/t/search-after-with-collapse-pagination-alternative/260451/1 "2021-01-07T14:33:37Z")

</div>

We want to utilize the _search\_after_ pagination strategy rather than _from-size_ pagination strategy as we understood that _search\_after_ has better performance for high pagination numbers.

At the same time, we do _collapse_ our query results base on specific field. We are aware of the limitation that _search\_after_ can't be applied when _collapse_ is utilized.

Question A: Based on [https://github.com/elastic/elasticsearch/issues/53115](https://github.com/elastic/elasticsearch/issues/53115), _composite aggregation_ can be used as an alternative to _from-size_ pagination - do I understand correctly that composite aggregation is used to retrieve page of "record ids", and second query is used to fetch records by ids returned from aggregation?

Question B: Are there other, performance wise better pagination approaches than _from-size_, that can be used together with field collapsing?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [January 7, 2021, 3:35pm UTC](https://discuss.elastic.co/t/search-after-with-collapse-pagination-alternative/260451/2 "2021-01-07T15:35:32Z")

</div>

> [@MilanGatyas](#):
>
> do I understand correctly that composite aggregation is used to retrieve page of "record ids", and second query is used to fetch records by ids returned from aggregation?

No, the challenge with doing any form of distributed aggregation with deep pagination is getting remote shards to independently agree on which subset of terms (not docs/doc IDs) to focus on with each request.

For scenarios where you want to do deep pagination there's broadly 3 approaches:

1. Composite agg - all shards work through results in order of a given grouping key (typically a field's terms e.g. customerID)).
2. Terms agg with partitioning - each shard works on an algorithmically determined subset of terms (uses the same sort of hash+modulo N partitioning approach used to evenly and deterministically index docs across shards by id)
3. Transforms API - fuses related docs together at index-time rather than the above query-time approaches.

This [wizard](https://plnkr.co/edit/eZr7r3KZW02AxNKAHCQa?p=preview&preview) can help walk through the options in more detail

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 4, 2021, 3:35pm UTC](https://discuss.elastic.co/t/search-after-with-collapse-pagination-alternative/260451/3 "2021-02-04T15:35:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.