# Why do my scroll operations fail?

**URL:** <https://discuss.elastic.co/t/why-do-my-scroll-operations-fail/864>\
**Category:** Elasticsearch\
**Created:** [May 18, 2015, 8:59pm UTC](https://discuss.elastic.co/t/why-do-my-scroll-operations-fail/864 "2015-05-18T20:59:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![ptrei](https://avatars.discourse-cdn.com/v4/letter/p/e480ec/32.png) [@ptrei](https://discuss.elastic.co/u/ptrei)\
**Post date:** [May 18, 2015, 8:59pm UTC](https://discuss.elastic.co/t/why-do-my-scroll-operations-fail/864/1 "2015-05-18T20:59:03Z")

</div>

Me again - I fixed my speed problem by moving to another system - I think the one I was on had flakey HW. But I'm still having problems with my small four node cluster; reading records keeps crahsing.

I'm using elasticsearch-py to read logstash-\* indexes, and generate secondary data. My indicies have up to 60 million records each. To read them, I'm using e-py's 'scan' helper:

```
import elasticsearch
import elasticsearch.helpers as helpers

es = elasticsearch.Elasticsearch(retry_on_timeout=True)
# sets up global ES handle

#main processing loop 
def process_index(index_name):
  global es
  query_body = '{"size": 10000, "query": {"match_all":{}}}' 
scanResp = helpers.scan(client=es,query=query_body,scroll="5m",index=index_name,timeout="5m")
  resp={}
  for resp in scanResp:
     DO STUFF FOR ONE RECORD

```

(yes, you saw this before)

My problem is that the reading fails after a while. I'm trying to understand if my code is wrong, my setup is wrong, or if its simply that my machines are underpowered:

Here's the crash, lightly redacted

```
Traceback (most recent call last):

```

File "pegasus5.py", line 580, in   
mymain()  
File "pegasus5.py", line 542, in mymain  
process\_index(indexname)  
File "pegasus5.py", line 414, in process\_index  
for resp in scanResp:  
File "build/bdist.linux-x86\_64/egg/elasticsearch/helpers/ **init**.py", line 230, in scan  
File "build/bdist.linux-x86\_64/egg/elasticsearch/client/utils.py", line 68, in \_wrapped  
File "build/bdist.linux-x86\_64/egg/elasticsearch/client/ **init**.py", line 616, in scroll  
File "build/bdist.linux-x86\_64/egg/elasticsearch/transport.py", line 308, in perform\_request  
File "build/bdist.linux-x86\_64/egg/elasticsearch/connection/http\_urllib3.py", line 86, in perform\_request  
File "build/bdist.linux-x86\_64/egg/elasticsearch/connection/base.py", line 102, in \_raise\_error  
elasticsearch.exceptions.NotFoundError: TransportError(404, u'{"\_scroll\_id":"c2NhbjswOzE7dG90YWxfaGl0czo0MDc1OTc4NDs=","took":709,"timed\_out":false,"\_shards":{"total":5,"successful":0,"failed":4,"failures":[{"status":404,  
"reason":"RemoteTransportException[[pegasus\_101][inet[/XXX.XXX.XXX.101:9300]][indices:data/read/search[phase/scan/scroll]]]; nested: SearchContextMissingException[No search context found for id [258]]; "},{"status":404,  
"reason":"RemoteTransportException[[pegasus-109][inet[/XXX.XXX.XXX.109:9300]][indices:data/read/search[phase/scan/scroll]]]; nested: SearchContextMissingException[No search context found for id [263]]; "},{"status":404,  
"reason":"RemoteTransportException[[pegasus\_101][inet[/XXX.XXX.XXX.101:9300]][indices:data/read/search[phase/scan/scroll]]]; nested: SearchContextMissingException[No search context found for id [257]]; "},{"status":404,  
"reason":"SearchContextMissingException[No search context found for id [325]]"}]},"hits":{"total":40759784,"max\_score":0.0,"hits":[]}}')

This program was running on node XXX.XXX.XXX.108

What is the reason behind a 'SearchContextMissingException? Are my systems timing out?

thanks  
Peter Trei

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:13am UTC](https://discuss.elastic.co/t/why-do-my-scroll-operations-fail/864/2 "2017-07-06T00:13:26Z")

</div>