# Scroll API nest for large datasets

**URL:** <https://discuss.elastic.co/t/scroll-api-nest-for-large-datasets/130062>\
**Category:** Elasticsearch\
**Created:** [April 30, 2018, 8:16pm UTC](https://discuss.elastic.co/t/scroll-api-nest-for-large-datasets/130062 "2018-04-30T20:16:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mww1](https://avatars.discourse-cdn.com/v4/letter/m/ccd318/32.png) [@mww1](https://discuss.elastic.co/u/mww1)\
**Post date:** [April 30, 2018, 8:16pm UTC](https://discuss.elastic.co/t/scroll-api-nest-for-large-datasets/130062/1 "2018-04-30T20:16:55Z")

</div>

Hello guys, I am using scan and scroll api for ES nest 5.5 to retrieve documents over 1.3 million in size(6 fields/document)  
my scroll api works fine if I retrieve less number of documents(around 800000) but when I want to retrieve large documents(1.3 million) it just fails. I dont know what is going wrong. I am trying to do this on C# visual studio and also make a web api and host as a microservice on service fabric app. both places it is failing. on visual studio, I get error: unexpected ES exception, sometimes I get web request aborted and sometimes " search context lost" even though I am giving new scrollid in each loop. below is my code. please help me. its urgent.

ISearchResponse initialResponse = this.ElasticClient.Search  
(scr =\> scr.Index(indexName)  
.From(0)  
.Take(scrollSize)  
.MatchAll()  
.Scroll(scrollTimeout)  
.searchtype(dfs));  
List results = new List();  
if (!initialResponse.IsValid || string.IsNullOrEmpty(initialResponse.ScrollId))  
throw new Exception(initialResponse.ServerError.Error.Reason);  
if (initialResponse.Documents.Any())  
results.AddRange(initialResponse.Documents);  
string scrollid = initialResponse.ScrollId;  
bool isScrollSetHasData = true;  
while (isScrollSetHasData)  
{  
ISearchResponse loopingResponse = this.ElasticClient.Scroll(scrollTimeout, scrollid);  
if (loopingResponse.IsValid)  
{  
results.AddRange(loopingResponse.Documents);  
scrollid = loopingResponse.ScrollId;  
}  
isScrollSetHasData = loopingResponse.Documents.Any();  
}  
this.ElasticClient.ClearScroll(new ClearScrollRequest(scrollid));

```auto
Please help

```

---

<div class="post-metadata">

**Author:** ![forloop](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forloop/32/9021_2.png) [@forloop](https://discuss.elastic.co/u/forloop)\
**Post date:** [May 1, 2018, 12:33pm UTC](https://discuss.elastic.co/t/scroll-api-nest-for-large-datasets/130062/2 "2018-05-01T12:33:30Z")

</div>



---

<div class="post-metadata">

**Author:** ![forloop](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forloop/32/9021_2.png) [@forloop](https://discuss.elastic.co/u/forloop)\
**Post date:** [May 1, 2018, 12:33pm UTC](https://discuss.elastic.co/t/scroll-api-nest-for-large-datasets/130062/3 "2018-05-01T12:33:51Z")

</div>

I'm closing this as it's a duplicate of [Scroll API(C# NEST) fails](https://discuss.elastic.co/t/scroll-api-c-nest-fails/129320/2)