How can we achieve the Pagination in ES Term Aggregation to avoid Memory issues.
Check out the use of "partitioning" in terms aggregation or try the composite aggregation with the after param.
The performance of both is slow. And when you do order by with them it's very slow.
Do you have numbers? Docs, ,nodes, response times
"partitioning" does not work the way Pagination is required.
The Requirement is like One aggregation has 10K records. Then i want to make that as order by on some param then i want the 1K record in next request i could be able to get the next bunch of records.
To achieve this we need one parameter as "FROM", size is already there and i know we can set the limit but to get next bunch of records it is not possible.
In a distributed system compromises are required - physics gets in the way.
I don’t understand the details of your query/data - please share a simple example of JSON using just the core fields you care about. Just show the query that gets you the top N and we’ll figure out how to get the next N after that.
Hi,
The pagination withe a From-Size approach is deprecated since Elasticsearch 5 if I remember well.
You should give a look to the scroll requests to slice the data.
Scroll is for retrieving docs, not aggregations (which are summaries derived from multiple docs).
Without seeing details of the query+data it's hard to determine if the required values are found in individual docs or combinations of multiple.
Indeed you're right.
So it strongly depend of the how the data structure across indices and mapping, and the repartition across shards and nodes.
Sorry for the interference.
Sorry for the interference.
No worries - scroll may still be the right answer. "It depends" as we like to say around here.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.