I have a index with around 50Million data points, where I have a ID for each document. I need unique id and its count , as it would be more than 10000, I used scrolling but unexpecteded, the scrolling gives me the same scrollid in the iteration > data = es.search(index="ttd-conversions-2019-05*", …

Don't use scroll. Use a regular search using the composite agg and then another search using a composite agg with the after parameter returned in the previous result. Repeat as necessary

Discuss the Elastic Stack

Scroll in ElasticSearch Aggregation

Elastic Stack Elasticsearch

Vinothkumar_Ganeshan (Vinothkumar Ganeshan) November 29, 2019, 3:53pm 7

Thanks for your time and effort. I would be helpful, if you could share your insights for the following problem Count in other index based on Current Index field

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.