I have a index with around 50Million data points, where I have a ID for each document. I need unique id and its count , as it would be more than 10000, I used scrolling but unexpecteded, the scrolling gives me the same scrollid in the iteration > data = es.search(index="ttd-conversions-2019-05*", …

Don't use scroll. Use a regular search using the composite agg and then another search using a composite agg with the after parameter returned in the previous result. Repeat as necessary

Scroll in ElasticSearch Aggregation

Elastic Stack Elasticsearch

Mark_Harwood (Mark Harwood) November 28, 2019, 4:49pm 2

The scroll api is used to scroll through documents, not aggregations.

To look at options for large numbers of unique terms try run this wizard to pick the right approach

1 Like

Topic		Replies	Views
About aggregations request with scroll, only the first results can be output? Elasticsearch	5	2304	February 24, 2018
How to scroll aggregation results using Java API Elasticsearch	3	1591	July 5, 2017
How to scroll through the returned aggregated results Elasticsearch	2	357	August 22, 2018
Elastic aggregation scrolling Elasticsearch	4	4643	July 5, 2017
Which SearchType will allow the aggregation during the searchScroll Elasticsearch	1	621	July 6, 2017

Scroll in ElasticSearch Aggregation

Related topics