Increase size limit - Python ElasticSearch

test_tester · February 19, 2021, 10:23am

Hi,

I'm currently writing a python script to extract out my elasticsearch documents and the following are my ES parameters that involves the sizing

response = es.search(index= es_index_list,scroll='5m', size='10000', body=search_index, request_timeout=60 )

As you can see my current size is 10000 and my python script can't extract anything beyond it. Is there a way to go about this?

Regards

warkolm · February 22, 2021, 12:55am

How many results do you want to get?

Sampson_Light · February 22, 2021, 3:24am

About 170k.

warkolm · February 22, 2021, 3:32am

Am I still talking to the same person here?

test_tester · February 22, 2021, 3:37am

Hello warkolm.

I have about 140,000 documents in one index which I was hoping to extract all in one go.

Regards

warkolm · February 22, 2021, 3:39am

Then don't use a search, use scan and scroll.

test_tester · February 22, 2021, 3:56am

Do you perhaps have an reference I can use to integrate into a python script?

warkolm · February 22, 2021, 4:02am

system · March 22, 2021, 4:02am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.