Increase size limit - Python ElasticSearch

Hi,

I'm currently writing a python script to extract out my elasticsearch documents and the following are my ES parameters that involves the sizing

response = es.search(index= es_index_list,scroll='5m', size='10000', body=search_index, request_timeout=60 )

As you can see my current size is 10000 and my python script can't extract anything beyond it. Is there a way to go about this?

Regards

1 Like

How many results do you want to get?

About 170k.

Am I still talking to the same person here? :stuck_out_tongue:

Hello warkolm.

I have about 140,000 documents in one index which I was hoping to extract all in one go.

Regards

Then don't use a search, use scan and scroll.

Do you perhaps have an reference I can use to integrate into a python script?

Elasticsearch and Scroll in Python | ObjectRocket looks like it should work.