How to increase the default size limit from 10000 to 1000000 in ElasticSearch

How to increase the default size limit from 10000 to 1000000 in ElasticSearch

Hi @chandu506 ,

Why don't you explore the scroll API?

https://www.elastic.co/guide/en/elasticsearch/reference/7.4/search-request-body.html#request-body-search-scroll

There are ways to increase to the default size limit. However, That is not recommended. An efficient way would be to use the scroll API and fetch the records page by page.

Hope that helps. Thanks.

Regards,
Somnath

1 Like

Welcome!

You can change index.max_result_window but you need to be careful when doing so as mentioned @somnath.guthula .

See documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html

Hi Somnath,

Thank you very much for your response.

Yes, I do agree with you comments and I have done the pagination on the web application, but we have a requirement where we pull certain amount of records like 1Million from ElasticSearch and send that data as an attachment in the email. So, in that case, I won't be able to do the pagination right, that's the reason why I had to increase the default max limit.

So, say if I increase the limit, will that impact all the searches even though we are use the pagination like from=&size=

Hope you understood my problem.

Thanks.
Chandra

Hi David,

Thank you very much for your response.

Yes, I do agree with you comments and I have done the pagination on the web application, but we have a requirement where we pull certain amount of records like 1Million from ElasticSearch and send that data as an attachment in the email. So, in that case, I won't be able to do the pagination right, that's the reason why I had to increase the default max limit.

So, say if I increase the limit, will that impact all the searches even though we are use the pagination like from=&size=

Hope you understood my problem.

Thanks.
Chandra

To extract data for further processing (here as an attachment), you need to use the _scroll API.