Extracting all search results from an index

(Anton Zakharov) #1


In our use case we need the ability to been able to get all search results for a particular index. The problem is that multiple users (some automated ones) may do this (or attempts to do this) at the same time. From my understanding i have the following possibilities:

  1. Use Scroll API. However from what i understand Scroll API allows 1 cursor by index at a time. Spliced scroll only accelerates the data transfer, by spicing the scroll, but does not allow 2 different queries to each run at the same time.

  2. Use Search After (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html ). My understanding is i can do multiple queries using search after at the same time. However it says in the documentation that : " the search_after parameter is stateless, it is always resolved against the latest version of the searcher. For this reason the sort order may change during a walk depending on the updates and deletes of your index". What exactly does it implies ?
    2.1 If somebody insert a document it may not appear ?
    2.2 If i delete a document and search after happens to have the deleted document id as a search after parameter.
    2.3 Is it possible to obtain duplicates value of the same document?

  3. What is the performance impact on using Search After versus Scroll API, especially if the number of documents returned is very large (millions+).

Thanks in advance

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.