# Question: Inconsistent count results **URL:** \ **Category:** Elasticsearch\ **Created:** [June 6, 2010, 11:13am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997 "2010-06-06T11:13:01Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Yatir\_Ben\_Shlomo\_3](https://avatars.discourse-cdn.com/v4/letter/y/b38774/32.png) [@Yatir\_Ben\_Shlomo\_3](https://discuss.elastic.co/u/Yatir_Ben_Shlomo_3)\ **Post date:** [June 6, 2010, 11:13am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/1 "2010-06-06T11:13:01Z")
Hi, Can someone explain why consecutive calls to the count REST API will give me different results ? Note that the results are swinging constantly between 2 numbers: 11340990 and 11341063 (needless to say I am not indexing during the calls to count ) curl -XGET '[http://ob1061:9200/en/organic/\_count?q=](http://ob1061:9200/en/organic/_count?q=)_:_' {"count":11340990,"\_shards":{"total":5,"successful":5,"failed":0}}( curl -XGET '[http://ob1061:9200/en/organic/\_count?q=](http://ob1061:9200/en/organic/_count?q=)_:_' {"count":11341063,"\_shards":{"total":5,"successful":5,"failed":0}} curl -XGET '[http://ob1061:9200/en/organic/\_count?q=](http://ob1061:9200/en/organic/_count?q=)_:_' {"count":11340990,"\_shards":{"total":5,"successful":5,"failed":0}} curl -XGET '[http://ob1061:9200/en/organic/\_count?q=](http://ob1061:9200/en/organic/_count?q=)_:_' {"count":11341063,"\_shards":{"total":5,"successful":5,"failed":0}} curl -XGET '[http://ob1061:9200/en/organic/\_count?q=](http://ob1061:9200/en/organic/_count?q=)_:_' ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 7, 2010, 8:05am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/2 "2010-06-07T08:05:11Z")
This should not be the case. Basically, count / search will round robin on the shards it search on, so you hit two different shards for consecutive requests, and it seems like a shard and its replica are not in sync. Did you hit any exceptions during indexing? On Sun, Jun 6, 2010 at 2:13 PM, Yatir Ben Shlomo [yatirb@gmail.com](mailto:yatirb@gmail.com) wrote: > Hi, > Can someone explain why consecutive calls to the count REST API will > give me different results ? > Note that the results are swinging constantly between 2 numbers: > 11340990 and 11341063 > (needless to say I am not indexing during the calls to count ) > > curl -XGET '[http://ob1061:9200/en/organic/\_count?q=\*:](http://ob1061:9200/en/organic/_count?q=*:)_' > {"count":11340990,"\_shards":{"total":5,"successful":5,"failed":0}}( > curl -XGET '[http://ob1061:9200/en/organic/\_count?q=\*:](http://ob1061:9200/en/organic/_count?q=*:)_' > {"count":11341063,"\_shards":{"total":5,"successful":5,"failed":0}} > curl -XGET '[http://ob1061:9200/en/organic/\_count?q=\*:](http://ob1061:9200/en/organic/_count?q=*:)_' > {"count":11340990,"\_shards":{"total":5,"successful":5,"failed":0}} > curl -XGET '[http://ob1061:9200/en/organic/\_count?q=\*:](http://ob1061:9200/en/organic/_count?q=*:)_' > {"count":11341063,"\_shards":{"total":5,"successful":5,"failed":0}} > curl -XGET '[http://ob1061:9200/en/organic/\_count?q=\*:](http://ob1061:9200/en/organic/_count?q=*:)\*' ---
**Author:** ![sakrafd](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@sakrafd](https://discuss.elastic.co/u/sakrafd)\ **Post date:** [October 22, 2011, 6:46pm UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/3 "2011-10-22T18:46:34Z")
I'm seeing the same thing: curl -X GET "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} curl -X GET "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} curl -X GET "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} curl -X GET "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} When I look at doc counts for each shard/replica (through elastic-search-head), i can see that one shard and replica is off: Primary: docs: { num\_docs: 3082198 max\_doc: 3545713 deleted\_docs: 463515 } Replica: docs: { num\_docs: 3081671 max\_doc: 3520572 deleted\_docs: 438901 } Is there a way to synch these without re-indexing? Also, just curious - what's the difference between num\_docs and max\_docs? Thanks, Dave ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 22, 2011, 7:34pm UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/4 "2011-10-22T19:34:39Z")
I can only answer to the last question. IMO, I suppose that max\_doc = num\_docs + deleted\_docs num\_docs = the current number of docs max\_doc = the maximum number of documents that ES had handle in the index David 😉 Le 22 oct. 2011 à 20:46, sakrafd [dave@signalhq.com](mailto:dave@signalhq.com) a écrit : > I'm seeing the same thing: > > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > > When I look at doc counts for each shard/replica (through > elastic-search-head), i can see that one shard and replica is off: > > Primary: > docs: { > num\_docs: 3082198 > max\_doc: 3545713 > deleted\_docs: 463515 > } > > Replica: > docs: { > num\_docs: 3081671 > max\_doc: 3520572 > deleted\_docs: 438901 > } > > Is there a way to synch these without re-indexing? > > Also, just curious - what's the difference between num\_docs and max\_docs? > > Thanks, > Dave > > -- > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html](http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 23, 2011, 12:17am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/5 "2011-10-23T00:17:23Z")
Are indexing going on when you execute the test? Can you try and refresh and see if you still get it? On Sat, Oct 22, 2011 at 8:46 PM, sakrafd [dave@signalhq.com](mailto:dave@signalhq.com) wrote: > I'm seeing the same thing: > > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > > When I look at doc counts for each shard/replica (through > elastic-search-head), i can see that one shard and replica is off: > > Primary: > docs: { > num\_docs: 3082198 > max\_doc: 3545713 > deleted\_docs: 463515 > } > > Replica: > docs: { > num\_docs: 3081671 > max\_doc: 3520572 > deleted\_docs: 438901 > } > > Is there a way to synch these without re-indexing? > > Also, just curious - what's the difference between num\_docs and max\_docs? > > Thanks, > Dave > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html](http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 23, 2011, 12:17am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/6 "2011-10-23T00:17:40Z")
Also, any failures in the log? On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Are indexing going on when you execute the test? Can you try and refresh > and see if you still get it? > > On Sat, Oct 22, 2011 at 8:46 PM, sakrafd [dave@signalhq.com](mailto:dave@signalhq.com) wrote: > > > I'm seeing the same thing: > > > > curl -X GET > > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > > > > When I look at doc counts for each shard/replica (through > > elastic-search-head), i can see that one shard and replica is off: > > > > Primary: > > docs: { > > num\_docs: 3082198 > > max\_doc: 3545713 > > deleted\_docs: 463515 > > } > > > > Replica: > > docs: { > > num\_docs: 3081671 > > max\_doc: 3520572 > > deleted\_docs: 438901 > > } > > > > Is there a way to synch these without re-indexing? > > > > Also, just curious - what's the difference between num\_docs and max\_docs? > > > > Thanks, > > Dave > > > > -- > > View this message in context: > > [http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html](http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html) > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Paul\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_brown/32/1449_2.png) [@Paul\_Brown](https://discuss.elastic.co/u/Paul_Brown)\ **Post date:** [October 23, 2011, 6:58am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/7 "2011-10-23T06:58:57Z")
I've seen the mismatched counts issue on a 4-node cluster, although it was only after some reconfiguration work that introduced temporary network partitions. Reindexing resolved the issue, and it hasn't recurred. Recovery from losing a node might eliminate the imbalance. What happens if you add &preference=\_primary or &preference=\_local to your query? -- Paul On Sat, Oct 22, 2011 at 11:46 AM, sakrafd [dave@signalhq.com](mailto:dave@signalhq.com) wrote: > I'm seeing the same thing: > > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23517808,"\_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742)" > {"count":23518180,"\_shards":{"total":10,"successful":10,"failed":0}} > > When I look at doc counts for each shard/replica (through > elastic-search-head), i can see that one shard and replica is off: > > Primary: > docs: { > num\_docs: 3082198 > max\_doc: 3545713 > deleted\_docs: 463515 > } > > Replica: > docs: { > num\_docs: 3081671 > max\_doc: 3520572 > deleted\_docs: 438901 > } > > Is there a way to synch these without re-indexing? > > Also, just curious - what's the difference between num\_docs and max\_docs? > > Thanks, > Dave > > -- > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html](http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![sakrafd](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@sakrafd](https://discuss.elastic.co/u/sakrafd)\ **Post date:** [October 23, 2011, 5:36pm UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/8 "2011-10-23T17:36:55Z")
Indexing is stopped and I see the same results after a refresh - the replica is off by 362 docs. The shard that is off - shard 0 - is on the master node. In those logs I see some recovery/rebalancing statements: [2011-10-18 19:42:53,551][DEBUG][gateway.local] [es\_db3] [users\_production][0]: allocating [[users\_production][0], node[null], [P], s[UNASSIGNED]] to [[es\_db3][3OIKkft6QeSGsjRE9TuIKg][inet[/:9300]]] on primary allocation [2011-10-18 19:43:31,655][DEBUG][action.admin.indices.status] [es\_db3] [users\_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P], s[INITIALIZING]: Failed to execute [org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@5cb27de5] org.elasticsearch.index.IndexShardMissingException: [users\_production][0] missing at org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:174) at org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:135) at org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:58) at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:231) at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:209) at org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:185) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) [2011-10-18 19:44:02,225][DEBUG][index.gateway] [es\_db3] [users\_production][0] starting recovery from local ... [2011-10-18 19:44:05,655][DEBUG][index.gateway] [es\_db3] [users\_production][0] recovery completed from local, took [3.4s] index : files [241] with total\_size [2.3gb], took[31ms] : recovered\_files [0] with total\_size [0b] : reusing\_files [241] with total\_size [2.3gb] translog : number\_of\_operations [0], took [3.4s] [2011-10-18 19:45:37,834][DEBUG][gateway.local] [es\_db3] [users\_production][0]: allocating [[users\_production][0], node[ null], [R], s[UNASSIGNED]] to [[es\_db0][4P-nUocJRTCrYQ5bH3TRUQ][inet[/:9300]]] in order to reuse its unallocated persistent store with total\_size [2.3gb] I'm seeing these for all the shards though. I also see similar things on the server with the replica. Is there something I should look for? Thanks for the help! Dave Shay Banon wrote: > Also, any failures in the log? > > On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) > [mailto:kimchy@gmail.com](mailto:kimchy@gmail.com)\> wrote: > > ``` > Are indexing going on when you execute the test? Can you try and > refresh and see if you still get it? > > On Sat, Oct 22, 2011 at 8:46 PM, sakrafd > wrote: > > I'm seeing the same thing: > > curl -X GET > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}} > > When I look at doc counts for each shard/replica (through > elastic-search-head), i can see that one shard and replica is off: > > Primary: > docs: { > num_docs: 3082198 > max_doc: 3545713 > deleted_docs: 463515 > } > > Replica: > docs: { > num_docs: 3081671 > max_doc: 3520572 > deleted_docs: 438901 > } > > Is there a way to synch these without re-indexing? > > Also, just curious - what's the difference between num_docs > and max_docs? > > Thanks, > Dave > > -- > View this message in context: > http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html > Sent from the ElasticSearch Users mailing list archive at > Nabble.com. > > ``` ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 24, 2011, 10:40pm UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/9 "2011-10-24T22:40:20Z")
You can restart the master node, thats fine, another one will be elected. Are there any other failures that happened? On Sun, Oct 23, 2011 at 7:36 PM, dave farkas [dave@signalhq.com](mailto:dave@signalhq.com) wrote: > Indexing is stopped and I see the same results after a refresh - the > replica is off by 362 docs. > > The shard that is off - shard 0 - is on the master node. In those logs I > see some recovery/rebalancing statements: > > [2011-10-18 19:42:53,551][DEBUG][gateway.\*\*local] [es\_db3] > [users\_production][0]: allocating [[users\_production][0], node[null], [P], > s[UNASSIGNED]] to [[es\_db3][**3OIKkft6QeSGsjRE9TuIKg][inet[/**:9300]]] > on primary allocation > > [2011-10-18 19:43:31,655][DEBUG][action.**admin.indices.status] [es\_db3] > [users\_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P], s[INITIALIZING]: > Failed to execute [org.elasticsearch.action.admin.indices.status. > IndicesStatusRequest@5cb27de5] > org.elasticsearch.index.IndexShardMissingException: > [users\_production][0] missing > at org.elasticsearch.index.service.InternalIndexService. > shardSafe(InternalIndexService.java:174) > at org.elasticsearch.action.admin.indices.status. > TransportIndicesStatusAction.shardOperation( > TransportIndicesStatusAction.java:135) > at org.elasticsearch.action.admin.indices.status. > TransportIndicesStatusAction.shardOperation( > TransportIndicesStatusAction.java:58) > at org.elasticsearch.action.support.broadcast. > TransportBroadcastOperationAction$AsyncBroadcastAction. > performOperation(TransportBroadcastOperationAction.java:231) > at org.elasticsearch.action.support.broadcast. > TransportBroadcastOperationAction$AsyncBroadcastAction. > performOperation(TransportBroadcastOperationAction.java:209) > at org.elasticsearch.action.support.broadcast. > TransportBroadcastOperationAct **ion$AsyncBroadcastAction$1.** run(** > TransportBroadcastOperationAct\*\*ion.java:185) > at java.util.concurrent.**ThreadPoolExecutor.runWorker(** > ThreadPoolExecutor.java:1110) > at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** > ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.\*\*java:636) > > [2011-10-18 19:44:02,225][DEBUG][index.\*\*gateway] [es\_db3] > [users\_production][0] starting recovery from local ... > [2011-10-18 19:44:05,655][DEBUG][index.\*\*gateway] [es\_db3] > [users\_production][0] recovery completed from local, took [3.4s] > index : files [241] with total\_size [2.3gb], took[31ms] > : recovered\_files [0] with total\_size [0b] > : reusing\_files [241] with total\_size [2.3gb] > translog : number\_of\_operations [0], took [3.4s] > [2011-10-18 19:45:37,834][DEBUG][gateway.\*\*local] [es\_db3] > [users\_production][0]: allocating [[users\_production][0], node[ > null], [R], s[UNASSIGNED]] to [[es\_db0][4P-\*_nUocJRTCrYQ5bH3TRUQ][inet[/\<_ > \*ip\>:9300]]] in order to reuse its unallocated persistent store with > total\_size [2.3gb] > > I'm seeing these for all the shards though. I also see similar things on > the server with the replica. Is there something I should look for? > > Thanks for the help! > Dave > > Shay Banon wrote: > > > Also, any failures in the log? > > > > On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) \ > [kimchy@gmail.com](mailto:kimchy@gmail.com)\>\> wrote: > > > > Are indexing going on when you execute the test? Can you try and > > refresh and see if you still get it? > > > > On Sat, Oct 22, 2011 at 8:46 PM, sakrafd \<[dave@signalhq.com](mailto:dave@signalhq.com) > > [mailto:dave@signalhq.com](mailto:dave@signalhq.com)\> wrote: > > > > ``` > > I'm seeing the same thing: > > > > curl -X GET > > "http://localhost:9200/users_ **production/user/_count?q=user_** > > > > ``` > > > > access\_id:742[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742) > > " > > {"count":23517808,"\_shards":{" **total":10,"successful":10,"** > > failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_\*\*production/user/\_count?q=user\_](http://localhost:9200/users_**production/user/_count?q=user_)\*\* > > access\_id:742[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742) > > " > > {"count":23518180,"\_shards":{" **total":10,"successful":10,"** > > failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_\*\*production/user/\_count?q=user\_](http://localhost:9200/users_**production/user/_count?q=user_)\*\* > > access\_id:742[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742) > > " > > {"count":23517808,"\_shards":{" **total":10,"successful":10,"** > > failed":0}} > > curl -X GET > > "[http://localhost:9200/users\_\*\*production/user/\_count?q=user\_](http://localhost:9200/users_**production/user/_count?q=user_)\*\* > > access\_id:742[http://localhost:9200/users\_production/user/\_count?q=user\_access\_id:742](http://localhost:9200/users_production/user/_count?q=user_access_id:742) > > " > > {"count":23518180,"\_shards":{" **total":10,"successful":10,"** > > failed":0}} > > > > ``` > > When I look at doc counts for each shard/replica (through > > elastic-search-head), i can see that one shard and replica is off: > > > > Primary: > > docs: { > > num_docs: 3082198 > > max_doc: 3545713 > > deleted_docs: 463515 > > } > > > > Replica: > > docs: { > > num_docs: 3081671 > > max_doc: 3520572 > > deleted_docs: 438901 > > } > > > > Is there a way to synch these without re-indexing? > > > > Also, just curious - what's the difference between num_docs > > and max_docs? > > > > Thanks, > > Dave > > > > -- > > View this message in context: > > http://elasticsearch-users. **115913.n3.nabble.com/Question-** > > > > ``` > > > > Inconsistent-count-results-\*\*tp873957p3444017.html[http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html](http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html) > > Sent from the Elasticsearch Users mailing list archive at > > [Nabble.com](http://Nabble.com). ---
**Author:** ![sakrafd](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@sakrafd](https://discuss.elastic.co/u/sakrafd)\ **Post date:** [October 26, 2011, 7:56pm UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/10 "2011-10-26T19:56:16Z")
Thanks! I didn't see any other types of failures - just ones like i referenced earlier. I ended up tracking down the missing docs and re-indexing them. On 10/24/11 5:40 PM, Shay Banon wrote: > You can restart the master node, thats fine, another one will be > elected. Are there any other failures that happened? > > On Sun, Oct 23, 2011 at 7:36 PM, dave farkas \<[dave@signalhq.com](mailto:dave@signalhq.com) > [mailto:dave@signalhq.com](mailto:dave@signalhq.com)\> wrote: > > ``` > Indexing is stopped and I see the same results after a refresh - > the replica is off by 362 docs. > > The shard that is off - shard 0 - is on the master node. In those > logs I see some recovery/rebalancing statements: > > [2011-10-18 19:42:53,551][DEBUG][gateway.local] > [es_db3] [users_production][0]: allocating [[users_production][0], > node[null], [P], s[UNASSIGNED]] to > [[es_db3][3OIKkft6QeSGsjRE9TuIKg][inet[/:9300]]] on primary > allocation > > [2011-10-18 19:43:31,655][DEBUG][action.admin.indices.status] > [es_db3] [users_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P], > s[INITIALIZING]: Failed to execute > [org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@5cb27de5] > org.elasticsearch.index.IndexShardMissingException: > [users_production][0] missing > at > org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:174) > at > org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:135) > at > org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:58) > at > org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:231) > at > org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:209) > at > org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:185) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:636) > > [2011-10-18 19:44:02,225][DEBUG][index.gateway] > [es_db3] [users_production][0] starting recovery from local ... > [2011-10-18 19:44:05,655][DEBUG][index.gateway] > [es_db3] [users_production][0] recovery completed from local, took > [3.4s] > index : files [241] with total_size [2.3gb], took[31ms] > : recovered_files [0] with total_size [0b] > : reusing_files [241] with total_size [2.3gb] > translog : number_of_operations [0], took [3.4s] > [2011-10-18 19:45:37,834][DEBUG][gateway.local] > [es_db3] [users_production][0]: allocating [[users_production][0], > node[ > null], [R], s[UNASSIGNED]] to > [[es_db0][4P-nUocJRTCrYQ5bH3TRUQ][inet[/:9300]]] in order to > reuse its unallocated persistent store with total_size [2.3gb] > > I'm seeing these for all the shards though. I also see similar > things on the server with the replica. Is there something I should > look for? > > Thanks for the help! > Dave > > Shay Banon wrote: > > Also, any failures in the log? > > On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon >> wrote: > > Are indexing going on when you execute the test? Can you > try and > refresh and see if you still get it? > > On Sat, Oct 22, 2011 at 8:46 PM, sakrafd > >> wrote: > > I'm seeing the same thing: > > curl -X GET > > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > > {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > > {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > > {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}} > curl -X GET > > "http://localhost:9200/users_production/user/_count?q=user_access_id:742" > > {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}} > > When I look at doc counts for each shard/replica (through > elastic-search-head), i can see that one shard and > replica is off: > > Primary: > docs: { > num_docs: 3082198 > max_doc: 3545713 > deleted_docs: 463515 > } > > Replica: > docs: { > num_docs: 3081671 > max_doc: 3520572 > deleted_docs: 438901 > } > > Is there a way to synch these without re-indexing? > > Also, just curious - what's the difference between num_docs > and max_docs? > > Thanks, > Dave > > -- > View this message in context: > http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html > Sent from the ElasticSearch Users mailing list archive at > Nabble.com. > > ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:50am UTC](https://discuss.elastic.co/t/question-inconsistent-count-results/2997/11 "2017-07-06T03:50:47Z")