# How to limit results with same field value to X amount of documents every time it shows up multiple times in a row **URL:** \ **Category:** Elasticsearch\ **Created:** [February 4, 2020, 1:22pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809 "2020-02-04T13:22:00Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![SeverinDK](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@SeverinDK](https://discuss.elastic.co/u/SeverinDK)\ **Post date:** [February 4, 2020, 1:22pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/1 "2020-02-04T13:22:00Z")
I have an issue that I am not quite sure how to solve. Really hope someone here can help me figure out how to go about it. Imagine that I have 100 documents, all with user\_id fields. I know that most documents are from different user\_ids, but documents 1-10 and 20-29 are from the same user\_id. What I want to do, is make sure that I only see the the latest two documents whenever a the same user\_id is returned in a row more than twice. So if user\_id 1 shows up more than twice in a row, I want to limit those documents. I want this to happen every time it happens for that user\_id, not limit it completely after that. * * * If I just request all documents as they are indexed now, I would get a result like: **[_1, 1, 1, 1, 1, 1, 1, 1, 1, 1_, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, _1, 1, 1, 1, 1, 1, 1, 1, 1, 1_, ...]** * * * What I am loooking for, us a way to make sure these groups of 1s, are limited to two documents in a row, like so: **[_1, 1_, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, _1, 1_, 12, ...]** Notice that `1, 1, ..., 1, 1, ...` happens here, meaning that the rows of identical user ids have been cut down to two, instead of removing them all together, which would result in something like: **[_1, 1_, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...]** * * * I also want this to work if the request is paginated (Multiple queries). So imagine that I request the first two pages, with a size of 5, then I would like to get: Page1: **[1, 1, 2, 3, 4]** Page2: **[5, 6, 7, 8, 9]** Instead of: Page1: **[1, 1, 2, 3, 4]** Page2: **[1, 1, 1, 1, 1]** * * * I hope that I have described the issue well enough for someone to understand. If not, then please let me know so I can try explaining it another way. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 4, 2020, 1:34pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/2 "2020-02-04T13:34:16Z")
Hi Søren, [Field collapsing](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-collapse) sounds like the closest thing to what you need. ---
**Author:** ![SeverinDK](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@SeverinDK](https://discuss.elastic.co/u/SeverinDK)\ **Post date:** [February 10, 2020, 10:45am UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/3 "2020-02-10T10:45:56Z")
Thank you for the answer Mark. I have looked into this, and it does seem like this is what I need. However, I am confused as to how I can paginate on this. If I set a size, that's just the top results, but the inner\_hits can have several results. So if I set an outer size of 10, there could really be up to 20 hits, as each top hit will have 1-2 inner hits. This would result in a weird response, when the client has requested 10, but get 20. Is there a common way around this that I am missing? ---
**Author:** ![wiouser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wiouser/32/45741_2.png) [@wiouser](https://discuss.elastic.co/u/wiouser)\ **Post date:** [February 11, 2020, 6:53am UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/4 "2020-02-11T06:53:33Z")
Just trying to understand the usecase. Are you using aggregations? Or just a plain query to match and retrieve the documents? Can your problem be described as `Latest 2 documents per user_id` But then user\_id won't appear again in the response if you use terms aggregation unless you use another sub-aggregation/field to group-by ---
**Author:** ![SeverinDK](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@SeverinDK](https://discuss.elastic.co/u/SeverinDK)\ **Post date:** [February 11, 2020, 9:20am UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/5 "2020-02-11T09:20:54Z")
Hi @wiouser - You understand my problem. I want the latest two documents, based on user\_id and never see it again on any other pages. I do not use any aggregation, just a plain query. ---
**Author:** ![wiouser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wiouser/32/45741_2.png) [@wiouser](https://discuss.elastic.co/u/wiouser)\ **Post date:** [February 11, 2020, 9:42am UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/6 "2020-02-11T09:42:34Z")
Then you have two options. If you want a limited number of user\_ids, you can use terms aggregation along with top\_hits as sub-aggregation. If you want to paginate results use composite aggregation along with top\_hits as sub-aggregation. I have given the terms and top\_hits aggregation example below. ``` GET myindex/_search { "size": 0, "aggs": { "user_list": { "terms": { "field": "user_id", "size": 10 }, "aggs": { "topdocuments": { "top_hits": { "size": 2, "sort": [{"time": {"order": "desc"}}] } } } } } } ``` I have sorted using time field to return the latest two documents. Take a look at top\_hits aggregation for more options. [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-top-hits-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-top-hits-aggregation.html) ---
**Author:** ![SeverinDK](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@SeverinDK](https://discuss.elastic.co/u/SeverinDK)\ **Post date:** [February 11, 2020, 3:42pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/7 "2020-02-11T15:42:39Z")
@wiouser Thank you for the answer. I tried it and it does indeed work, but as with the solution that @Mark_Harwood suggested, here, I also end up with 10 groups with 1-2 items per user. So what confuses me is if I want to paginate on this result, the client will request `?page=1&limit=10`, but in the aggregated result set, I have 10-20 items, so I would have to discard some of it, to fulfill the promise of `limit=10` I might be missing some basic ES knowledge here ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [February 11, 2020, 5:42pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/8 "2020-02-11T17:42:20Z")
> [@SeverinDK](#): > > I would have to discard some of it, Discarding results might not be the end of the world. We often discard matches behind the scenes when pulling the top N of something from a distributed set of data stores. ---
**Author:** ![SeverinDK](https://avatars.discourse-cdn.com/v4/letter/s/ac8455/32.png) [@SeverinDK](https://discuss.elastic.co/u/SeverinDK)\ **Post date:** [February 12, 2020, 11:10am UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/9 "2020-02-12T11:10:24Z")
For my case I cannot discard it. We're building a feed with users latest two posts. A user can upload multiple posts in a row, but we do not want the feed flooded by the same user, if they upload 10+ posts in one go, so we want to make sure it's only the last two. So for every page, I would have to decide on a few things that would now get shown. It would be random if I just start picking things out on each request, to make sure there are only 10 per page. Not sure what my options are here and if ES can help me, out of the box. ---
**Author:** ![wiouser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wiouser/32/45741_2.png) [@wiouser](https://discuss.elastic.co/u/wiouser)\ **Post date:** [February 13, 2020, 12:58pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/10 "2020-02-13T12:58:21Z")
Try `terms` value source in composite aggregation instead of terms aggregation. You have to send `after_key` for every subsequent request. It also support `size` and `order` and you can have top\_hits as sub aggregation fior composite as in the previous example. [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-composite-aggregation.html) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 12, 2020, 12:58pm UTC](https://discuss.elastic.co/t/how-to-limit-results-with-same-field-value-to-x-amount-of-documents-every-time-it-shows-up-multiple-times-in-a-row/217809/11 "2020-03-12T12:58:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.