I have a index with around 50Million data points, where I have a ID for each document. I need unique id and its count , as it would be more than 10000, I used scrolling but unexpecteded, the scrolling gives me the same scrollid in the iteration > data = es.search(index="ttd-conversions-2019-05*", …

Don't use scroll. Use a regular search using the composite agg and then another search using a composite agg with the after parameter returned in the previous result. Repeat as necessary

Scroll in ElasticSearch Aggregation

Elastic Stack Elasticsearch

Vinothkumar_Ganeshan (Vinothkumar Ganeshan) November 29, 2019, 3:53pm 7

Thanks for your time and effort. I would be helpful, if you could share your insights for the following problem Count in other index based on Current Index field

Topic		Replies	Views
Scroll id is not changing while querying Elasticsearch	2	4885	December 8, 2017
Retrieving millions of large documents Elasticsearch	7	364	September 25, 2023
Same scroll id Elasticsearch	3	1778	July 5, 2017
Help needed with Proper usage of scroll api using python: Getting the same results Elasticsearch	5	2099	July 13, 2018
Scroll to get scroll_id only Elasticsearch	1	609	August 23, 2017

Scroll in ElasticSearch Aggregation

Related Topics