ES query for supporting pagination

Elastic Stack Elasticsearch
1

I have data in ES as:

"hits": {
    "total": 177,
    "max_score": 1,
    "hits": [
            {
             "_id": "random_id_1",
             "_score": 1,
             "_source": {
                 "title": "Find series",
                 "description": "some random text",
                 "createdAt": 1527848716000,
                 "category": [
                     "STARS"
                 ],
                 "updatedAt": 1527848716000
               }
            },
            {
             "_id": "random_id_2",
             "_score": 1,
             "_source": {
                 "title": "Find series 2",
                 "description": "some random text 2",
                 "createdAt": 1527848716001,
                 "category": [
                     "SOLARS"
                 ],
                 "updatedAt": 1527848716001
               }
            }
         ]

What is wanted is to display n data from each category. For which query formed using aggregation as:

    {
	"size": 0,
    "aggs": {
    "summary": {
      "terms": {
        "field": "category"
      },                                                              
      "aggs": {                                                          
        "summary": {                                             
          "top_hits": {                                                  
            "sort": [                                                    
              {                                                        
                "createdAt": "desc"                                     
              }                                                        
            ],
            "size": 100
          }                                                            
        }                                                              
      }
    }
  }
}

But what i am unable to get is pagination. If i use from under nested aggregation, i can have pagination, but if data gets updated, from will be of no use. What i want is keyset pagination based on _id, but am not able to figure out how to achieve the same? Any help in here. Thanks

2

Pagination of the top-level terms agg (you're only seeing the first 10 here) or of the top hits under each term?

i can have pagination, but if data gets updated, from will be of no use.

This is always the case. Only the scroll api supports paging over a fixed "point-in-time" view of data while updates are being received and that API only supports paging through docs.

What i want is keyset pagination based on _id,

The doc _ids? I'm not sure that makes sense as your top_hits are sorted by date and your terms aggs are the top N terms by popularity.

Perhaps we should start with what business problem you're trying to solve?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.