Get all ids with Python

Elastic Stack Elasticsearch
1

Hi,
I'd like to use the elastic_enterprise_search.AppSearch package to get all our document ids. I have tried many things and always hit the 10k result limit. I understand that "scrolling" may be the solution but have not seen it work. This is last thing I tried:

es = AppSearch(APP_SEARCH_API_HOST, bearer_auth=APP_SEARCH_API_KEY)
body = {"size": 100, "scroll": "1m"}
scroll_response = es.search(engine_name=ENGINE, body=body)
document_ids = []
hits = scroll_response.get("results", [])
for hit in hits:
    document_ids.append(hit["_id"])

Please advise.
Thank you.

2

Have you used "from" keyword:
Example


i = 0
document_ids = []
while i<100000:
   body = { "from": i }
   scroll_response = es.search(engine_name=ENGINE, body=body)
   i+=10000
   for hit in hits:
          document_ids.append(hit["_id"])
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.