Elasticsearch SearchContextMissingException during 'scan & scroll' query with Spring Data Elasticsearch

Elastic Stack Elasticsearch
1

I am using elasticsearch 2.2.0 with default cluster configuration. I
encounter a problem with scan and scroll query using spring data
elasticsearch. When I execute the query I get error like this:

[2016-06-29 12:45:52,046][DEBUG][action.search.type       ] [Vector] [155597] Failed to execute query phase
RemoteTransportException[[Vector][10.132.47.95:9300][indices:data/read/search[phase/scan/scroll]]]; nested: SearchContextMissingException[No search context found for id [155597]];
Caused by: SearchContextMissingException[No search context found for id [155597]]
    at org.elasticsearch.search.SearchService.findContext(SearchService.java:611)
    at org.elasticsearch.search.SearchService.executeScan(SearchService.java:311)
    at org.elasticsearch.search.action.SearchServiceTransportAction$SearchScanScrollTransportHandler.messageReceived(SearchServiceTransportAction.java:433)
    at org.elasticsearch.search.action.SearchServiceTransportAction$SearchScanScrollTransportHandler.messageReceived(SearchServiceTransportAction.java:430)
    at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

My 'scan & scroll' code:

public List<T> getAllElements(SearchQuery searchQuery) {
    searchQuery.setPageable(new PageRequest(0, PAGE_SIZE));
    String scrollId = elasticsearchTemplate.scan(searchQuery, 1000, false);
    List<T> allElements = new LinkedList<>();
    boolean hasRecords = true;
    while (hasRecords) {
        Page<T> page = elasticsearchTemplate.scroll(scrollId, 5000, resultMapper);
        if (page.hasContent()) {
            allElements.addAll(page.getContent());
        } else {
            hasRecords = false;
        }
    }
    elasticsearchTemplate.clearScroll(scrollId);
    return allElements;
}

When my query result size is less than PAGE_SIZE parameter, then
error like this occurs five times. I guess that it is one per shard.
When result size is bigger than PAGE_SIZE, the error occurs few times
more. I've tried to refactor my code to not call:

Page<T> page = elasticsearchTemplate.scroll(scrollId, 5000, resultMapper);

when I'm sure that the page has no content. But it works only if
PAGE_SIZE is bigger than query result, so it is no the solution at all.

I have to add that it is problem only on elasticsearch side. On the
client side the errors is hidden and in each case the query result is
correct. Has anybody knows what causes this issue?

Thank you for help,

Simon.

2

This usually happens if your search context is not alive anymore.

In your case, you're starting your scan with a timeout of 1 second and then during each scan the search context is kept alive during 5 seconds. It's probably too low. The default duration to keep the search context alive is 1 minute, so you should probably increase it to 60 seconds like this:

String scrollId = elasticsearchTemplate.scan(searchQuery, 60000, false);
...
Page<T> page = elasticsearchTemplate.scroll(scrollId, 60000, resultMapper);