# SearchHits.totalHits() vs. SearchHits.getHits().length

**URL:** <https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059>\
**Category:** Elasticsearch\
**Created:** [October 22, 2013, 5:10pm UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059 "2013-10-22T17:10:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alexander\_Kley](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@Alexander\_Kley](https://discuss.elastic.co/u/Alexander_Kley)\
**Post date:** [October 22, 2013, 5:10pm UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/1 "2013-10-22T17:10:38Z")

</div>

Hi guys,

im feeling strange, because query my test index deliver different amount of  
results.

I have an index A with three types A1, A2 and A3. All entries have the same  
source -\> FIELD01:"This is a smal text". Every type hat 100 entries.

If i query the whole index with a lucene native query string like this  
FIELD01:"This is a smal text" via prepareSearch() on the client and  
SearchType.SCAN, i got 300 hits as result of  
SearchResponse.getHits().totalHits();

Then, if i scroll through the result via prepareSearchScroll() i got everey  
time if i start the test another length of SearchHit[] Array.

Where is my problem? Why deliver SearchResp.getHits().totalHits() 300  
and SearchResp.getHits().getHits().length every run another amount \< 300?

For better understanding my small code snip:

SearchResponse scrollResp =  
client.prepareSearch(index).setTypes(types).setSearchType(SearchType.SCAN)  
.setScroll(new  
TimeValue(60000)).setQuery(QueryBuilders.queryString(luceneQuery))  
.setSize(50).execute().actionGet();

while (true)  
{  
scrollResp =  
this.esClient.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new  
TimeValue(60000)).execute().actionGet();  
logger.debug("[searchByLuceneQuery] [{}] record(s) found in  
scrolling searchResponse", scrollResp.getHits().totalHits()); // -\> everey  
run 300

for (final SearchHit hit : scrollResp.getHits())  
{  
if (!hit.isSourceEmpty())  
{  
......

}

if (scrollResp.getHits().getHits().length == 0) // -\> never 300, every time  
\< 300  
{  
break;  
}  
}  
}

Best regards  
Alex

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Alexander\_Kley](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@Alexander\_Kley](https://discuss.elastic.co/u/Alexander_Kley)\
**Post date:** [October 22, 2013, 5:24pm UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/2 "2013-10-22T17:24:40Z")

</div>

T[http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-td1575408.html](http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-td1575408.html)here  
is a post from 2010 with similarly behavior.  
[http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-td1575408.html](http://elasticsearch-users.115913.n3.nabble.com/totalHits-gets-changed-unexpectedly-while-scrolling-SearchResponse-td1575408.html)

Is it still a problem?

Am Dienstag, 22. Oktober 2013 19:10:38 UTC+2 schrieb Alex:

> Hi guys,
> 
> im feeling strange, because query my test index deliver different amount  
> of results.
> 
> I have an index A with three types A1, A2 and A3. All entries have the  
> same source -\> FIELD01:"This is a smal text". Every type hat 100 entries.
> 
> If i query the whole index with a lucene native query string like this  
> FIELD01:"This is a smal text" via prepareSearch() on the client and  
> SearchType.SCAN, i got 300 hits as result of  
> SearchResponse.getHits().totalHits();
> 
> Then, if i scroll through the result via prepareSearchScroll() i got  
> everey time if i start the test another length of SearchHit Array.
> 
> Where is my problem? Why deliver SearchResp.getHits().totalHits() 300  
> and SearchResp.getHits().getHits().length every run another amount \< 300?
> 
> For better understanding my small code snip:
> 
> SearchResponse scrollResp =  
> client.prepareSearch(index).setTypes(types).setSearchType(SearchType.SCAN)  
> .setScroll(new  
> TimeValue(60000)).setQuery(QueryBuilders.queryString(luceneQuery))  
> .setSize(50).execute().actionGet();
> 
> while (true)  
> {  
> scrollResp =  
> this.esClient.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new  
> TimeValue(60000)).execute().actionGet();  
> logger.debug("[searchByLuceneQuery] [{}] record(s) found in  
> scrolling searchResponse", scrollResp.getHits().totalHits()); // -\> everey  
> run 300
> 
> for (final SearchHit hit : scrollResp.getHits())  
> {  
> if (!hit.isSourceEmpty())  
> {  
> ......
> 
> }
> 
> if (scrollResp.getHits().getHits().length == 0) // -\> never 300, every  
> time \< 300  
> {  
> break;  
> }  
> }  
> }
> 
> Best regards  
> Alex

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [October 22, 2013, 5:59pm UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/3 "2013-10-22T17:59:15Z")

</div>

Please note, you requested with setSize() the number of results to cache  
per shard. You have used 50, so you will only get 300 hits in a scroll  
response if you have 6 shards.

totalHits() is not related to scrolling at all, but to the whole result set  
size.

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Alexander\_Kley](https://avatars.discourse-cdn.com/v4/letter/a/6a8cbe/32.png) [@Alexander\_Kley](https://discuss.elastic.co/u/Alexander_Kley)\
**Post date:** [October 22, 2013, 6:20pm UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/4 "2013-10-22T18:20:37Z")

</div>

Thanks Jörg for yout hint. I have tried some other approaches and have  
found a good resolution for me. I think the problem was the searchType  
'SCAN'. Now I use the default one and it works fine.

scrollResp =  
this.esClient.prepareSearch(index).setTypes(types).setQuery(QueryBuilders.queryString(luceneQuery)).setSize(fetchSizeInternal).setScroll(new  
TimeValue(60000)).execute().actionGet();

while (scrollResp.getHits().getHits().length \> 0)  
{  
for (final SearchHit hit : scrollResp.getHits().getHits())  
{  
//do some with the hit  
}

```
    scrollResp = 

```

this.esClient.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new  
TimeValue(60000)).execute().actionGet();  
}

Am Dienstag, 22. Oktober 2013 19:59:15 UTC+2 schrieb Jörg Prante:

> Please note, you requested with setSize() the number of results to cache  
> per shard. You have used 50, so you will only get 300 hits in a scroll  
> response if you have 6 shards.
> 
> totalHits() is not related to scrolling at all, but to the whole result  
> set size.
> 
> Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Boaz\_Leskes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boaz_leskes/32/723_2.png) [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)\
**Post date:** [October 23, 2013, 9:10am UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/5 "2013-10-23T09:10:36Z")

</div>

Hi Alex,

I'm not sure what you are trying to achieve but typically you'd use  
scrolling to go through a complete result set. In that case the SCAN search  
type is the one you want as it is way more efficient with large result sets  
(with the price of giving up sorting).

With scrolling you basically iterate over the set of documents that matches  
your query by reading smaller sets of documents at a time. Think about  
going through 1M documents, reading 200 at a time. scrollResp.totalHits()  
returns the total number of document that matches your query (1M in my  
example) and scrollResp.getHits().getHits().length is the size of the chunk  
you currently retrieved (200 in my example). As jorg mentioned, the size of  
that chunk is what ever you've set in the setSize() parameter multiplied by  
the number of shards an index have.

Cheers,  
Boaz

On Tuesday, October 22, 2013 8:20:37 PM UTC+2, Alex wrote:

> Thanks Jörg for yout hint. I have tried some other approaches and have  
> found a good resolution for me. I think the problem was the searchType  
> 'SCAN'. Now I use the default one and it works fine.
> 
> scrollResp =  
> this.esClient.prepareSearch(index).setTypes(types).setQuery(QueryBuilders.queryString(luceneQuery)).setSize(fetchSizeInternal).setScroll(new  
> TimeValue(60000)).execute().actionGet();
> 
> while (scrollResp.getHits().getHits().length \> 0)  
> {  
> for (final SearchHit hit : scrollResp.getHits().getHits())  
> {  
> //do some with the hit  
> }
> 
> ```
> scrollResp = 
> 
> ```
> 
> this.esClient.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new  
> TimeValue(60000)).execute().actionGet();  
> }
> 
> Am Dienstag, 22. Oktober 2013 19:59:15 UTC+2 schrieb Jörg Prante:
> 
> > Please note, you requested with setSize() the number of results to cache  
> > per shard. You have used 50, so you will only get 300 hits in a scroll  
> > response if you have 6 shards.
> > 
> > totalHits() is not related to scrolling at all, but to the whole result  
> > set size.
> > 
> > Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:11am UTC](https://discuss.elastic.co/t/searchhits-totalhits-vs-searchhits-gethits-length/14059/6 "2017-07-06T02:11:03Z")

</div>