Search API Limits

Elastic Stack Elasticsearch
1

Hi all!

I'm new to elastic and also new to the forum, so please forgive me if I ask something stupid or wrong.
I have a problem using Search API of Java High Level REST Client because I'm trying to make a request with "from" parameter set with value 1000000000 and the "size" parameter equal to 100 (this is a test case obviously), but when i try to make the search request I receive the following error:

[org.elasticsearch.client.ResponseException: method [POST], host [http://127.0.0.1:9200], URI [/test_idx/test_type/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [1000000100]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"test_idx","node":"jifB2vzCRM6N7aOb1c8YSg","reason":{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [1000000100]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}}]},"status":500}]

I know that elastic has a default value of 10000 for max_result_window, but in this case I want 100 elements (as specified with the size parameter) starting from 1000000000. Is my reasoning wrong? Could you tell me if maybe I don't understood well how "from" and "size" works?

Thank you in advance

2

You are right, and as it's mentioned in the documentation there's a limitation of 10000.
But it doesn't matter the number of size, because the limitation is on the addition of : from + size

Note that from + size can not be more than the index.max_result_window index setting which defaults to 10,000

3

@klof Ok but assume that I have 1000000 documents in my index and I want to retrieve the last 10 elements, how can I do if from+size must be less then the aforementioned limitation?

Thanks

4

Maybe you could :

I don't know if these approaches can be applied in your case?

2 Likes
5

I think that scroll API is the only solution. Thank you for your help!

6

There is also search_after. Scroll might be what you want though, because it has a consistent snapshot.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.