# Total Hits in elastic Search prepareSearch() behaves in an inconsistent manner
**URL:** https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963
**Category:** Elasticsearch
**Created:** [October 4, 2018, 4:39am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963 "2018-10-04T04:39:49Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 4, 2018, 4:39am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/1 "2018-10-04T04:39:49Z")
I'm new to ElasticSearch and my scenario is something like, there is an index with some 100 documents. I'm using following snippet to retrieve those
ListenableActionFuture f = esClient.prepareSearch(index) .setTypes(type) .setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH) .setQuery(multiMatchQuery) .execute();
I want to fetch all documents at one shot without using scroll. I know that index wont be having so many records. If I specify
ListenableActionFuture f = esClient.prepareSearch(index) .setTypes(type) .setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH) .setQuery(multiMatchQuery) .setSize(100) .execute(); behaviour is inconsistent. Number of records getting fetched varies every time, sometimes 34 documents, sometimes 21 etc.
But if I dont specify size, its fetching 10 always. I'm using elasticsearch 5.6.3.
I have tried few solutions such as using without search type as below,but no luck
ListenableActionFuture f = esClient.prepareSearch(index) .setTypes(type) .setQuery(multiMatchQuery) .setSize(hitcount) .execute();
I need a query to fetch all records at one shot. When I try to print searchresponse,I found total hits is incorrect as below
hits": { "total": 25, "max\_score": 0.18232156, "hits": [ { .... } }} It should be 100. Again if I execute, same code, I'm getting different total hits.
My index is having 5 shards
I'm wondering what makes me getting inconsistent total hits often.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 4, 2018, 4:54am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/2 "2018-10-04T04:54:17Z")
Just set size to 10000.
BTW you probably need only one shard.
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 4, 2018, 5:07am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/3 "2018-10-04T05:07:19Z")
I tried to set 1000, but now the total hits roaming around 35, sometimes 36, 34, etc
My total match hit size should be 73.
My updated code
ListenableActionFuture f = esClient.prepareSearch(index)
.setTypes(type)
.setQuery(multiMatchQuery)
.setSize(10000)
.execute();
"BTW you probably need only one shard.",
which means in case of multiple shards, how to read all matching records at one shot?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 4, 2018, 12:13pm UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/4 "2018-10-04T12:13:40Z")
Could you share the exact REST call and result and the exact Java code (ie what is the query, ...)?
I just meant that if you have not that much data, one shard should be enough.
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 4, 2018, 12:52pm UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/5 "2018-10-04T12:52:41Z")
Here is the complete code
```
String[] stringArray= new String[1];
stringArray[0]="status";
MultiMatchQueryBuilder multiMatchQuery = QueryBuilders.multiMatchQuery("running",stringArray);
ListenableActionFuture f = esClient.getESClient().prepareSearch(index)
.setTypes(type)
.setQuery(multiMatchQuery)
.setSize(10000)
.execute();
```
I may have max 5000 records in this index. In that case having single shard will have any performance or any other issue? If no, how to specify number of shards in java api while creating index?
In case I prefer to use 5 shards, is there no way to get all records at one shot?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 4, 2018, 2:17pm UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/6 "2018-10-04T14:17:12Z")
> Could you share the exact REST call and result?
I need that as well.
> I may have max 5000 records in this index. In that case having single shard will have any performance or any other issue?
That will be definitely better with one single shard. Faster.
You define the number of shards when creating the index by providing index settings.
Example here: [Create Index API | Java REST Client [7.17] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-create-index.html#_index_settings)
> In case I prefer to use 5 shards, is there no way to get all records at one shot?
This is not related to the problem you exposed.
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 5, 2018, 4:52am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/7 "2018-10-05T04:52:06Z")
> [@dadoonet](#):
>
> > Could you share the exact REST call and result?
I'm invoking SearchRequestBuilder through Java API to fetch results.I have written only below code, where esclient.getESClient() will return me
org.elasticsearch.client.Client and I will append the required parameters to that.
ListenableActionFuture f = esClient.getESClient().prepareSearch(index)
.setTypes(type)
.setQuery(multiMatchQuery)
.setSize(10000)
.execute();
ListenableActionFuture f is having below result
{
"took": 2,
"timed\_out": false,
"\_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 7,
"max\_score": 0.2876821,
"hits": [
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"_id": "AWZCfKD-3sx8bE5KQlR_",
"\_score": 0.2876821,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "realtestindex",
"typename": "health"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKED3sx8bE5KQlSA",
"\_score": 0.2876821,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "realtestindex",
"typename": "sessions"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKFE3sx8bE5KQlSC",
"\_score": 0.2876821,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "testindex",
"typename": "a071971a-8645-4abf-b367-e4eeadd777e6"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKFI3sx8bE5KQlSE",
"\_score": 0.2876821,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "testindex",
"typename": "19ae3b6b-cad2-40d4-84c3-c23b40edca87"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKDt3sx8bE5KQlR-",
"\_score": 0.13353139,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "realtestindex",
"typename": "events"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKFC3sx8bE5KQlSB",
"\_score": 0.13353139,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "testindex",
"typename": "46d75306-a891-43b2-b80a-760749f1fb3c"
}
},
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZCfKFF3sx8bE5KQlSD",
"\_score": 0.13353139,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "testindex",
"typename": "564815a7-56c1-41b7-a3a5-cfc07b453fdc"
}
}
]
}
}
Please let me know if you need more details
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 5, 2018, 6:05am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/8 "2018-10-05T06:05:46Z")
I changed my index to have single shard
{"reindexjob":{"settings":{"index":{"creation\_date":"1538717770126","number\_of\_shards":"1","number\_of\_replicas":"1","uuid":"cNYWZtkCRCeNv\_-8jM124A","version":{"created":"5060399"},"provided\_name":"reindexjob"}}}}
But still total hits are inconsistent. When I query the count via rest call
curl -XGET 'localhost:9200/reindexjob/\_count' -d '{"query" : {"match" : {"status":"running"}}}'
I'm getting 73 as result.
{"count":73,"\_shards":{"total":1,"successful":1,"skipped":0,"failed":0}},
same when I'm doing via java call
SearchRequest{searchType=QUERY\_THEN\_FETCH, indices=[reindexjob], indicesOptions=IndicesOptions[id=38, ignore\_unavailable=false, allow\_no\_indices=true, expand\_wildcards\_open=true, expand\_wildcards\_closed=false, allow\_alisases\_to\_multiple\_indices=true, forbid\_closed\_indices=true], types=[scrolljob], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, source={
"size" : 10000,
"query" : {
"multi\_match" : {
"query" : "running",
"fields" : [
"status^1.0"
],
"type" : "best\_fields",
"operator" : "OR",
"slop" : 0,
"prefix\_length" : 0,
"max\_expansions" : 50,
"lenient" : false,
"zero\_terms\_query" : "NONE",
"boost" : 1.0
}
}
}}
I'm getting inconsistent responses as above.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 5, 2018, 6:26am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/9 "2018-10-05T06:26:23Z")
Can you try the exact same query?
Either match on both tests or multimatch?
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 5, 2018, 7:20am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/10 "2018-10-05T07:20:53Z")
I tried to make make curl call sync with java API at the maximum as below
curl -XGET 'localhost:9200/reindexjob/\_search?search\_type=query\_then\_fetch&size=10000' -d '{"query" : {"multi\_match" : {"query":"running","fields":["status"],"type":"best\_fields", "operator" : "OR", "slop" : 0, "prefix\_length" : 0, "max\_expansions" : 50, "lenient" : false, "zero\_terms\_query" : "NONE", "boost" : 1.0}}}'
result is 73 always as below
{
"took": 2,
"timed\_out": false,
"\_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 73,
"max\_score": 0.006779687,
"hits": [
{
"\_index": "reindexjob",
"\_type": "scrolljob",
"\_id": "AWZC9ULQ3sx8bE5KQlue",
"\_score": 0.006779687,
"\_source": {
"scrollid": null,
"status": "running",
"indexname": "testindex",
"typename": "69625b89-5095-4964-94f8-ae9a60025653"
}
},
....... here 72 more records .....
}
]
}
}
Additional details, if I do remote debugging on this java call , then my response is having 73 records in java layer always ..
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 9, 2018, 6:31am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/11 "2018-10-09T06:31:21Z")
Can you please provide any update on this?
---
### Author:  [@johnrabi](https://discuss.elastic.co/u/johnrabi)
#### Post date: [October 9, 2018, 6:46am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/12 "2018-10-09T06:46:29Z")
> [@RAM\_NATHAN](#):
>
> MultiMatchQueryBuilder multiMatchQuery = QueryBuilders.multiMatchQuery("running",stringArray); ListenableActionFuture\ f = esClient.getESClient().prepareSearch(index)
Hi @RAM_NATHAN,
need the exact count of specified fields values means can you use match\_phrase query with field names
curl -X GET "localhost:9200/\_search" -H 'Content-Type: application/json' -d'
{
"query": {
"match\_phrase" : {
"message" : "this is a test"
}
}
}
'
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 12, 2018, 5:41am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/13 "2018-10-12T05:41:19Z")
Thanks for the update. But using match\_phrase, will solve my problem?
Should MultiMatchQueryBuilder not be used, in case of getting count?
Is this way match\_phrase should be used?
QueryBuilder queryBuilder = QueryBuilders.matchPhraseQuery("\_all", "Test Type Search");
SearchResponse response = client.prepareSearch("index\_name")
.setQuery(queryBuilder)
.setFrom(1)
.setSize(10)
.execute()
.actionGet();
---
### Author:  [@johnrabi](https://discuss.elastic.co/u/johnrabi)
#### Post date: [October 12, 2018, 6:04am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/14 "2018-10-12T06:04:21Z")
Hi @RAM_NATHAN,
You need to get the total records and count means please use matchAllQuery ,
need to get one field value (like status ="running") document count means use matchphrasequery ("fieldname","value");
QueryBuilder queryBuilder = QueryBuilders.matchPhraseQuery("\_all", "Test Type Search");
SearchResponse response = client.prepareSearch("index\_name")
.setQuery(queryBuilder)
.setFrom(1)
.setSize(10)
.execute()
.actionGet();
---
### Author:  [@RAM\_NATHAN](https://discuss.elastic.co/u/RAM_NATHAN)
#### Post date: [October 15, 2018, 11:21am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/15 "2018-10-15T11:21:15Z")
> [@johnrabi](#):
>
> QueryBuilder queryBuilder = QueryBuilders.matchPhraseQuery("\_all", "Test Type Search");
MatchPhrase too cant help. Again inconsistent results some times 28, sometimes 10.
please find the response below.
{
"took": 0,
"timed\_out": false,
"\_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 22,
"max\_score": 0.021978907,
"hits": [..........]
}
}
I'm using reactive pattern. Does it make any difference?
Observable.create(new OnSubscribe\\>() {
@Override
public void call(Subscriber\\> t) {
```
QueryBuilder queryBuilder = QueryBuilders.matchPhraseQuery(filterKey,filterValue);
ListenableActionFuture f = esClient.getESClient().prepareSearch(index)
.setQuery(queryBuilder).
setFrom(0)
.setSize(10000)
.execute();
Observable.from(f).map(x -> {
List> innerSearchHits = new ArrayList>();
SearchHits hits = x.getHits();
for (SearchHit searchHit : hits) {
.............
}
return innerSearchHits;
}).onErrorReturn(e -> {
......
return null;
}).subscribe(...
}
}, e -> {
if (!t.isUnsubscribed()) {
t.onError(e);
}
}, () -> {
if (!t.isUnsubscribed()) {
t.onCompleted();
}
});
}
});
```
Same code is working fine, if I do remote debugging
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 12, 2018, 11:21am UTC](https://discuss.elastic.co/t/total-hits-in-elastic-search-preparesearch-behaves-in-an-inconsistent-manner/150963/16 "2018-11-12T11:21:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.