Way to get all documents more than 10,000,000

Elastic Stack Elasticsearch
1

Dears, I want to get all docuemnts in an index.
I already know the way but there was a limitation.

Way to get documents more than 10,000.

  • ES supprt size prametter. But the maximum value is 10,000.
  • So to get more than 10,000 documents, Need use sort with search_after. Can navigate page.
    Paginate search results | Reference

Limitation

My index has over 10,000,000 documents.
First time, above way worked. But when I kept navigating another page. I got memory full errors. And can't use ES anymore.

Is there other efficient way? I think sort use lots of memory.
I have to get all huge documents from index..

Thank you very much

2

How many documents do you need to retrieve per pagination?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.