# 0 hits, 14693688 total on index with 700 documents
**URL:** https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608
**Category:** Elasticsearch
**Created:** [October 3, 2017, 8:07pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608 "2017-10-03T20:07:19Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 3, 2017, 8:07pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/1 "2017-10-03T20:07:19Z")
Hi, I'm having the following problem on an Elasticsearch 5.4.3 cluster. When making a query where some shards don't return any hits they seem to return an incorrect total. This happens only on some nodes and restarting them fixed it temporarily.
```
/_search?q=status:any
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 32,
"successful": 32,
"failed": 0
},
"hits": {
"total": 14693688,
"max_score": 0.0,
"hits": [
]
}
}
```
The same query might return total: 0, if it doesn't go to any of the problematic nodes. I've tried hitting a certain shard with preference=\_shards:n|\_local and the problem seems to be at the node level, it also happens in every index in the cluster.
The Count API with similar queries returns correct totals.
Any help is appreciated!
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 6, 2017, 5:43am UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/2 "2017-10-06T05:43:17Z")
What's the output of `\_cat/shards?v? look like?
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 6, 2017, 2:37pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/3 "2017-10-06T14:37:05Z")
Looks normal
```
21 r STARTED 24 110.6kb
21 r STARTED 24 110.6kb
21 p STARTED 24 110kb
12 p STARTED 32 100.8kb
12 r STARTED 32 100.8kb
12 r STARTED 32 100.8kb
```
All shards are STARTED.
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 6, 2017, 8:35pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/4 "2017-10-06T20:35:45Z")
Do you have an index with 21 shards?
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 9, 2017, 4:08pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/5 "2017-10-09T16:08:06Z")
32 actually (probably a bad idea in this case). Do you think the number of shards is related to the issue?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 9, 2017, 7:18pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/6 "2017-10-09T19:18:50Z")
If you have 32 shards for 700 docs it's never a good idea.
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 9, 2017, 10:46pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/7 "2017-10-09T22:46:19Z")
I agree.
Anyhow, I don't think the problem is related to the number of shards. To test it out I created a new index with 5 shards and 1 replica in the same cluster, indexed 2 documents and sent a similar query:
```
curl -X POST /test-totals/_search -d '{"query": {"term": {"status": "created"}}}'
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1836712,
"max_score": 0.2876821,
"hits": [
{
"_index": "test-totals",
"_type": "test",
"_id": "1",
"_score": 0.2876821,
"_source": {
"status": "created"
}
}
]
}
}
```
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 9, 2017, 11:07pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/8 "2017-10-09T23:07:36Z")
Thing is, the number of docs in a shard influence the results of relevance in a search.
I would 100% recommend you try again with a single shard.
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 10, 2017, 6:07am UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/9 "2017-10-10T06:07:07Z")
But something looks weird here. How can you get that number of hits with only 2 documents indexed?
Could you share the full script to reproduce that? I feel like you are doing something wrong.
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 10, 2017, 2:16pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/10 "2017-10-10T14:16:40Z")
Hi, I'm afraid you will not be able to reproduce it easily, I have several clusters running elastic 5.4.3 and I only see this issue in this one. As I said in my original post, I believe it is an issue at the node level as it happens on shards that are in a couple of nodes and it is fixed temporarily when restarting those nodes. It then starts happening on other nodes.
What I did was:
```
curl -X PUT http://localhost:9200/test-totals -d '{"number_of_shards": 5}'
curl -X PUT http://localhost:9200/test-totals/test/1 -d '{"status": "created"}'
curl -X PUT http://localhost:9200/test-totals/test/2 -d '{"status": "other"}'
curl -X POST http://localhost:9200/test-totals/_search -d '{"query": {"term": {"status": "created"}}}'
```
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 10, 2017, 2:31pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/11 "2017-10-10T14:31:29Z")
As Mark asked, can you run:
```auto
GET _cat/shards?v
```
And share all the output?
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 10, 2017, 3:23pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/12 "2017-10-10T15:23:01Z")
Sure: [shards.txt](https://s3.amazonaws.com/public-es-data/shards.txt).
It's a bit of a mess but I have an identical cluster with the same data and indices that doesn't have this behavior.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 10, 2017, 3:38pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/13 "2017-10-10T15:38:39Z")
I did not see any `test-totals` index in the shards output. Is that by any chance an alias? If so, what is the definition of that alias?
---
### Author:  [@spappier](https://discuss.elastic.co/u/spappier)
#### Post date: [October 10, 2017, 6:03pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/14 "2017-10-10T18:03:34Z")
Didn't realize I had deleted the index after the test, here is the output of every step:
```
curl -X PUT http://localhost:9200/test-totals
# {"acknowledged":true,"shards_acknowledged":true}
curl -X PUT http://localhost:9200/test-totals/test/1 -d '{"status": "created"}'
# {"_index":"test-totals","_type":"test","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"created":true}
curl -X PUT http://localhost:9200/test-totals/test/2 -d '{"status": "other"}'
# {"_index":"test-totals","_type":"test","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":2,"failed":0},"created":true}
curl -X POST http://localhost:9200/test-totals/_search -d '{"query": {"term": {"status": "created"}}}'
# {"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1836712,"max_score":0.2876821,"hits":[{"_index":"test-totals","_type":"test","_id":"1","_score":0.2876821,"_source":{"status": "created"}}]}}
curl -X GET http://localhost:9200/_cat/shards/test-totals
# test-totals 1 r STARTED 0 130b 10.64.93.143 i-0a24bef506fa1166b
# test-totals 1 p STARTED 0 130b 10.64.95.71 i-02a3e2df248b271d5
# test-totals 2 p STARTED 1 3.2kb 10.64.92.161 i-09cab816e090dfea9
# test-totals 2 r STARTED 1 3.2kb 10.64.94.22 i-0344657cb1aecd18b
# test-totals 4 p STARTED 0 130b 10.64.92.230 i-0c9b38f6ebf2a6658
# test-totals 4 r STARTED 0 130b 10.64.93.252 i-0ac7a100527b0fde2
# test-totals 3 p STARTED 1 3.2kb 10.64.92.212 i-05122a8f025abd4e1
# test-totals 3 r STARTED 1 3.2kb 10.64.95.83 i-0fab00d6d8bee2e58
# test-totals 0 p STARTED 0 130b 10.64.94.131 i-004eab7c862e21279
# test-totals 0 r STARTED 0 130b 10.64.92.77 i-0687677d1d8b2eb11
```
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [October 10, 2017, 6:36pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/15 "2017-10-10T18:36:21Z")
Any chance you could share the `data` dir of your cluster and send it by email?
I'm super curious about that behaviour.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 7, 2017, 6:37pm UTC](https://discuss.elastic.co/t/0-hits-14693688-total-on-index-with-700-documents/102608/16 "2017-11-07T18:37:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.