### Author:  [@Alex\_Wajda](https://discuss.elastic.co/u/Alex_Wajda)
#### Post date: [September 24, 2010, 5:17pm UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/1 "2010-09-24T17:17:46Z")
Hi guys,
I'm scrolling the query result using the approach described here
>
(elasticsearch version 0.10.0)
For some reasons the total hits number unexpectedly decreases during
scrolling which of course leads to the processing less data than
expected.
Here's my code snippet on Scala which does all this work:
```
val scrollKeepAliveTime = new TimeValue(5, TimeUnit.MINUTES)
var searchResponse = esClient.prepareSearch(INDEX_NAME).
setTypes(myType).
setQuery(myBoolQuery).
setScroll(scrollKeepAliveTime).
execute.actionGet
while (searchResponse.hits.hits.length > 0) {
for (hit <- searchResponse.hits.hits) {
// Take 'hit' and do some magic
}
searchResponse =
```
esClient.prepareSearchScroll(searchResponse.scrollId).
setScroll(scrollKeepAliveTime).
execute.actionGet
}
Example:
When the searchResponse is obtained first time by executing
esClient.prepareSearch(...) it shows the total hits number 14. The
default scroll size is 10, so I process first 10 hits and request the
next portion of data via
esClient.prepareSearchScroll(searchResponse.scrollId), but instead of
expected 4 hits I get only 2 and 'searchResponse.hits().totalHits' now
shows only 12 hits (instead of 14).
And this behaviour is not 100% repeatable. Sometimes 14 becomes 12,
sometimes 11 and rare I get all 14.
Am I doing something wrong?
And another question - inside the for() loop in the above code "do
some magic" is going to mean - to make some changes to the retrieved
document and re-index it:
esClient.prepareIndex(INDEX\_NAME, myType, hit.id).
setSource(updatedDocument).
execute.actionGet
Will this work well during scrolling? I mean won't the modifications
make any conflicts to the scrolling "cursor"?
Thank you!
Alex.
---
### Author:  [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)
#### Post date: [September 24, 2010, 5:25pm UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/2 "2010-09-24T17:25:02Z")
Hiya
> I'm scrolling the query result using the approach described here
Scrolling is still broken for me:
> **[Issues · elastic/elasticsearch](https://github.com/elastic/elasticsearch/issues#issue/136)**
>
> Free and Open, Distributed, RESTful Search Engine. Contribute to elastic/elasticsearch development by creating an account on GitHub.
I'd avoid it for now. You can replicate its function using the 'from'
parameter.
clint
---
### Author:  [@DKichler](https://discuss.elastic.co/u/DKichler)
#### Post date: [October 3, 2010, 8:13pm UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/3 "2010-10-03T20:13:28Z")
I am experiencing this scrolling error as well with code following the same
pattern as your example. It has possibly been introduced in 0.9.0 because I
wrote the code against 0.8 and seem to remember it working as I expected.
Looking closer at the SearchResponse returned by the first scrolling
operation, it consistently shows a single shard failure (4 out of 5
successful shards) with an exception message like this:
shard [\_na], reason [SearchContextMissingException[No search context found
for id [5], timed out]]
It seems to me as though the failed shard could be holding the entries that
mysteriously disappear after the first scroll.
On Fri, Sep 24, 2010 at 10:25 AM, Clinton Gormley [via Elasticsearch Users]
\<[ml-node+1575442-286360478-88438@n3.nabble.com](mailto:ml-node+1575442-286360478-88438@n3.nabble.com)[ml-node%2B1575442-286360478-88438@n3.nabble.com](mailto:ml-node%2B1575442-286360478-88438@n3.nabble.com)
> wrote:
> Hiya
>
> > I'm scrolling the query result using the approach described here
>
> Scrolling is still broken for me:
> [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues#issue/136)
>
> I'd avoid it for now. You can replicate its function using the 'from'
> parameter.
>
> clint
>
> * * *
>
> View message @
> [http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-tp1575408p1575442.html](http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-tp1575408p1575442.html)
> To start a new topic under Elasticsearch Users, email
> [ml-node+115913-2004534304-88438@n3.nabble.com](mailto:ml-node+115913-2004534304-88438@n3.nabble.com)[ml-node%2B115913-2004534304-88438@n3.nabble.com](mailto:ml-node%2B115913-2004534304-88438@n3.nabble.com)
> To unsubscribe from Elasticsearch Users, click here[http://elasticsearch-users.115913.n3.nabble.com/template/TplServlet.jtp?tpl=unsubscribe\_by\_code&node=115913&code=ZGtpY2hsZXJAc2Vla2Vyc29sLmNhfDExNTkxM3wtNTUwNDk3NDU5](http://elasticsearch-users.115913.n3.nabble.com/template/TplServlet.jtp?tpl=unsubscribe_by_code&node=115913&code=ZGtpY2hsZXJAc2Vla2Vyc29sLmNhfDExNTkxM3wtNTUwNDk3NDU5).
---
### Author:  [@Tomislav\_Poljak](https://discuss.elastic.co/u/Tomislav_Poljak)
#### Post date: [October 11, 2010, 7:07pm UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/4 "2010-10-11T19:07:16Z")
Hi,
it seems (to me) scrolling doesn't work as expected (not retrieving all
matched documents) when using Java API on index with more than one shard
(for a single shard index seems to work fine).
Is there any chance this issue will be addressed in the near future or
is 'from' parameter workaround a way to go?
Also, can you see any changes done after you started the first 'scroll'
when using 'from' parameter workaround (or is it the same as scrolling)?
Tomislav
On Fri, 2010-09-24 at 19:25 +0200, Clinton Gormley wrote:
> Hiya
>
> > I'm scrolling the query result using the approach described here
>
> Scrolling is still broken for me:
> [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues#issue/136)
>
> I'd avoid it for now. You can replicate its function using the 'from'
> parameter.
>
> clint
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [October 11, 2010, 7:17pm UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/5 "2010-10-11T19:17:20Z")
Hey,
Yea, I plan to address the scrolling issue. Regarding
the visibility aspect, its a point in time scroll, so you won't see any
changes happening after the first search+scroll request has been executed.
-shay.banon
On Mon, Oct 11, 2010 at 9:07 PM, Tomislav Poljak [tpoljak@gmail.com](mailto:tpoljak@gmail.com) wrote:
> Hi,
> it seems (to me) scrolling doesn't work as expected (not retrieving all
> matched documents) when using Java API on index with more than one shard
> (for a single shard index seems to work fine).
>
> Is there any chance this issue will be addressed in the near future or
> is 'from' parameter workaround a way to go?
>
> Also, can you see any changes done after you started the first 'scroll'
> when using 'from' parameter workaround (or is it the same as scrolling)?
>
> Tomislav
>
> On Fri, 2010-09-24 at 19:25 +0200, Clinton Gormley wrote:
>
> > Hiya
> >
> > > I'm scrolling the query result using the approach described here
> >
> > Scrolling is still broken for me:
> > [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues#issue/136)
> >
> > I'd avoid it for now. You can replicate its function using the 'from'
> > parameter.
> >
> > clint
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:18am UTC](https://discuss.elastic.co/t/totalhits-gets-changed-unexpectedly-while-scrolling-searchresponse/3366/6 "2017-07-06T04:18:07Z")