# Query by max date

**URL:** <https://discuss.elastic.co/t/query-by-max-date/251424>\
**Category:** Elasticsearch\
**Created:** [October 8, 2020, 11:10am UTC](https://discuss.elastic.co/t/query-by-max-date/251424 "2020-10-08T11:10:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kennethtruyers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kennethtruyers/32/21056_2.png) [@Kennethtruyers](https://discuss.elastic.co/u/Kennethtruyers)\
**Post date:** [October 8, 2020, 11:10am UTC](https://discuss.elastic.co/t/query-by-max-date/251424/1 "2020-10-08T11:10:55Z")

</div>

I have an index that is append only. Every change gets saved as a new document, together with a date property. In order to get the current active data I need to group all entities by their ID and get the one with the latest date. It should be possible to add a filter for that date. I have been playing around with aggregations but I got stuck with the scroll API.

Data example (note: id's are guids, dates are unix timestamps, using other values here to make it clearer)

```auto
   {"id": "<entityId_date>", "entityId", 1, "date": "2020-05-10", "value": { <JSON document> } }, 
   {"id": "<entityId_date>", "entityId", 1, "date": "2020-06-10", "value": { <JSON document> } }, 
   {"id": "<entityId_date>", "entityId", 2, "date": "2020-04-10", "value": { <JSON document> } }, 
   {"id": "<entityId_date>", "entityId", 2, "date": "2020-08-10", "value": { <JSON document> } },
   {"id": "<entityId_date>", "entityId", 3, "date": "2020-07-10", "value": { <JSON document> } }

```

Required results:

```auto
   {"id": "<entityId_date>", "entityId", 1, "date": "2020-06-10", "value": { <JSON document> } }, 
   {"id": "<entityId_date>", "entityId", 2, "date": "2020-08-10", "value": { <JSON document> } },
   {"id": "<entityId_date>", "entityId", 3, "date": "2020-07-10", "value": { <JSON document> } }

```

Tried the following query:

```auto
{
  "size": 0,
  "aggs": {
    "id_aggregate": {
      "terms": { 
          "field": "entityId"
       },
      "aggs": {
        "date_aggregate": {
          "top_hits": {
            "sort": [{
                "date.ticks": {
                    "order": "desc"
                }
            }],
            "size": 1
          }
        }
     }
    }
  }
}

```

The result I get back is correct, but I only get the first X results back. I can set the size of the `id_aggregate`, but there could be thousands of documents.  
With a normal search query, I'd be able to use the Scroll API, but this is not possible with aggregations.

I have looked into using partitions, but that doesn't help me either, because sometimes there will be very few documents and sometimes there will be loads. (This is kind of a base query, all other queries we execute will have additional filters specified).

How can I build a query that fetches documents by their max date and is scrollable?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2020, 11:10am UTC](https://discuss.elastic.co/t/query-by-max-date/251424/2 "2020-11-05T11:10:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.