Question: Inconsistent count results

Yatir_Ben_Shlomo_3 · June 6, 2010, 11:13am

Hi,
Can someone explain why consecutive calls to the count REST API will
give me different results ?
Note that the results are swinging constantly between 2 numbers:
11340990 and 11341063
(needless to say I am not indexing during the calls to count )

curl -XGET 'http://ob1061:9200/en/organic/_count?q=:'
{"count":11340990,"_shards":{"total":5,"successful":5,"failed":0}}(
curl -XGET 'http://ob1061:9200/en/organic/_count?q=:'
{"count":11341063,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=:'
{"count":11340990,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=:'
{"count":11341063,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=:'

kimchy · June 7, 2010, 8:05am

This should not be the case. Basically, count / search will round robin on
the shards it search on, so you hit two different shards for consecutive
requests, and it seems like a shard and its replica are not in sync. Did you
hit any exceptions during indexing?

On Sun, Jun 6, 2010 at 2:13 PM, Yatir Ben Shlomo yatirb@gmail.com wrote:

Hi,
Can someone explain why consecutive calls to the count REST API will
give me different results ?
Note that the results are swinging constantly between 2 numbers:
11340990 and 11341063
(needless to say I am not indexing during the calls to count )

curl -XGET 'http://ob1061:9200/en/organic/_count?q=*:'
{"count":11340990,"_shards":{"total":5,"successful":5,"failed":0}}(
curl -XGET 'http://ob1061:9200/en/organic/_count?q=*:'
{"count":11341063,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=*:'
{"count":11340990,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=*:'
{"count":11341063,"_shards":{"total":5,"successful":5,"failed":0}}
curl -XGET 'http://ob1061:9200/en/organic/_count?q=*:*'

sakrafd · October 22, 2011, 6:46pm

I'm seeing the same thing:

curl -X GET "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

When I look at doc counts for each shard/replica (through elastic-search-head), i can see that one shard and replica is off:

Primary:
docs: {
num_docs: 3082198
max_doc: 3545713
deleted_docs: 463515
}

Replica:
docs: {
num_docs: 3081671
max_doc: 3520572
deleted_docs: 438901
}

Is there a way to synch these without re-indexing?

Also, just curious - what's the difference between num_docs and max_docs?

Thanks,
Dave

dadoonet · October 22, 2011, 7:34pm

I can only answer to the last question.

IMO, I suppose that
max_doc = num_docs + deleted_docs

num_docs = the current number of docs
max_doc = the maximum number of documents that ES had handle in the index

David

Le 22 oct. 2011 à 20:46, sakrafd dave@signalhq.com a écrit :

I'm seeing the same thing:

curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

When I look at doc counts for each shard/replica (through
elastic-search-head), i can see that one shard and replica is off:

Primary:
docs: {
num_docs: 3082198
max_doc: 3545713
deleted_docs: 463515
}

Replica:
docs: {
num_docs: 3081671
max_doc: 3520572
deleted_docs: 438901
}

Is there a way to synch these without re-indexing?

Also, just curious - what's the difference between num_docs and max_docs?

Thanks,
Dave

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

kimchy · October 23, 2011, 12:17am

Are indexing going on when you execute the test? Can you try and refresh and
see if you still get it?

On Sat, Oct 22, 2011 at 8:46 PM, sakrafd dave@signalhq.com wrote:

I'm seeing the same thing:

curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

When I look at doc counts for each shard/replica (through
elastic-search-head), i can see that one shard and replica is off:

Primary:
docs: {
num_docs: 3082198
max_doc: 3545713
deleted_docs: 463515
}

Replica:
docs: {
num_docs: 3081671
max_doc: 3520572
deleted_docs: 438901
}

Is there a way to synch these without re-indexing?

Also, just curious - what's the difference between num_docs and max_docs?

Thanks,
Dave

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

kimchy · October 23, 2011, 12:17am

Also, any failures in the log?

On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon kimchy@gmail.com wrote:

Are indexing going on when you execute the test? Can you try and refresh
and see if you still get it?

On Sat, Oct 22, 2011 at 8:46 PM, sakrafd dave@signalhq.com wrote:

I'm seeing the same thing:

curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

When I look at doc counts for each shard/replica (through
elastic-search-head), i can see that one shard and replica is off:

Primary:
docs: {
num_docs: 3082198
max_doc: 3545713
deleted_docs: 463515
}

Replica:
docs: {
num_docs: 3081671
max_doc: 3520572
deleted_docs: 438901
}

Is there a way to synch these without re-indexing?

Also, just curious - what's the difference between num_docs and max_docs?

Thanks,
Dave

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

Paul_Brown · October 23, 2011, 6:58am

I've seen the mismatched counts issue on a 4-node cluster, although it
was only after some reconfiguration work that introduced temporary
network partitions. Reindexing resolved the issue, and it hasn't
recurred. Recovery from losing a node might eliminate the imbalance.

What happens if you add &preference=_primary or &preference=_local to
your query?

-- Paul

On Sat, Oct 22, 2011 at 11:46 AM, sakrafd dave@signalhq.com wrote:

I'm seeing the same thing:

curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
curl -X GET
"http://localhost:9200/users_production/user/_count?q=user_access_id:742"
{"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

When I look at doc counts for each shard/replica (through
elastic-search-head), i can see that one shard and replica is off:

Primary:
docs: {
num_docs: 3082198
max_doc: 3545713
deleted_docs: 463515
}

Replica:
docs: {
num_docs: 3081671
max_doc: 3520572
deleted_docs: 438901
}

Is there a way to synch these without re-indexing?

Also, just curious - what's the difference between num_docs and max_docs?

Thanks,
Dave

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

sakrafd · October 23, 2011, 5:36pm

Indexing is stopped and I see the same results after a refresh - the
replica is off by 362 docs.

The shard that is off - shard 0 - is on the master node. In those logs I
see some recovery/rebalancing statements:

[2011-10-18 19:42:53,551][DEBUG][gateway.local ] [es_db3]
[users_production][0]: allocating [[users_production][0], node[null],
[P], s[UNASSIGNED]] to
[[es_db3][3OIKkft6QeSGsjRE9TuIKg][inet[/:9300]]] on primary allocation

[2011-10-18 19:43:31,655][DEBUG][action.admin.indices.status] [es_db3]
[users_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P],
s[INITIALIZING]: Failed to execute
[org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@5cb27de5]
org.elasticsearch.index.IndexShardMissingException:
[users_production][0] missing
at
org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:174)
at
org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:135)
at
org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:58)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:231)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:209)
at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:185)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

[2011-10-18 19:44:02,225][DEBUG][index.gateway ] [es_db3]
[users_production][0] starting recovery from local ...
[2011-10-18 19:44:05,655][DEBUG][index.gateway ] [es_db3]
[users_production][0] recovery completed from local, took [3.4s]
index : files [241] with total_size [2.3gb], took[31ms]
: recovered_files [0] with total_size [0b]
: reusing_files [241] with total_size [2.3gb]
translog : number_of_operations [0], took [3.4s]
[2011-10-18 19:45:37,834][DEBUG][gateway.local ] [es_db3]
[users_production][0]: allocating [[users_production][0], node[
null], [R], s[UNASSIGNED]] to
[[es_db0][4P-nUocJRTCrYQ5bH3TRUQ][inet[/:9300]]] in order to reuse
its unallocated persistent store with total_size [2.3gb]

I'm seeing these for all the shards though. I also see similar things on
the server with the replica. Is there something I should look for?

Thanks for the help!
Dave

Shay Banon wrote:

Also, any failures in the log?

On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon <kimchy@gmail.com
mailto:kimchy@gmail.com> wrote:

Are indexing going on when you execute the test? Can you try and
refresh and see if you still get it?


On Sat, Oct 22, 2011 at 8:46 PM, sakrafd <dave@signalhq.com
<mailto:dave@signalhq.com>> wrote:

    I'm seeing the same thing:

     curl -X GET
    "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
    {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
     curl -X GET
    "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
    {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
     curl -X GET
    "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
    {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
     curl -X GET
    "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
    {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

    When I look at doc counts for each shard/replica (through
    elastic-search-head), i can see that one shard and replica is off:

    Primary:
    docs: {
       num_docs: 3082198
       max_doc: 3545713
       deleted_docs: 463515
    }

    Replica:
    docs: {
       num_docs: 3081671
       max_doc: 3520572
       deleted_docs: 438901
    }

    Is there a way to synch these without re-indexing?

    Also, just curious - what's the difference between num_docs
    and max_docs?

    Thanks,
    Dave

    --
    View this message in context:
    http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
    Sent from the ElasticSearch Users mailing list archive at
    Nabble.com.

kimchy · October 24, 2011, 10:40pm

You can restart the master node, thats fine, another one will be elected.
Are there any other failures that happened?

On Sun, Oct 23, 2011 at 7:36 PM, dave farkas dave@signalhq.com wrote:

Indexing is stopped and I see the same results after a refresh - the
replica is off by 362 docs.

The shard that is off - shard 0 - is on the master node. In those logs I
see some recovery/rebalancing statements:

[2011-10-18 19:42:53,551][DEBUG][gateway.**local ] [es_db3]
[users_production][0]: allocating [[users_production][0], node[null], [P],
s[UNASSIGNED]] to [[es_db3][3OIKkft6QeSGsjRE9TuIKg][inet[/:9300]]]
on primary allocation

[2011-10-18 19:43:31,655][DEBUG][action.admin.indices.status] [es_db3]
[users_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P], s[INITIALIZING]:
Failed to execute [org.elasticsearch.action.admin.indices.status.
IndicesStatusRequest@5cb27de5]
org.elasticsearch.index.IndexShardMissingException:
[users_production][0] missing
at org.elasticsearch.index.service.InternalIndexService.
shardSafe(InternalIndexService.java:174)
at org.elasticsearch.action.admin.indices.status.
TransportIndicesStatusAction.shardOperation(
TransportIndicesStatusAction.java:135)
at org.elasticsearch.action.admin.indices.status.
TransportIndicesStatusAction.shardOperation(
TransportIndicesStatusAction.java:58)
at org.elasticsearch.action.support.broadcast.
TransportBroadcastOperationAction$AsyncBroadcastAction.
performOperation(TransportBroadcastOperationAction.java:231)
at org.elasticsearch.action.support.broadcast.
TransportBroadcastOperationAction$AsyncBroadcastAction.
performOperation(TransportBroadcastOperationAction.java:209)
at org.elasticsearch.action.support.broadcast.
TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(
TransportBroadcastOperationAct**ion.java:185)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.**java:636)

[2011-10-18 19:44:02,225][DEBUG][index.**gateway ] [es_db3]
[users_production][0] starting recovery from local ...
[2011-10-18 19:44:05,655][DEBUG][index.**gateway ] [es_db3]
[users_production][0] recovery completed from local, took [3.4s]
index : files [241] with total_size [2.3gb], took[31ms]
: recovered_files [0] with total_size [0b]
: reusing_files [241] with total_size [2.3gb]
translog : number_of_operations [0], took [3.4s]
[2011-10-18 19:45:37,834][DEBUG][gateway.**local ] [es_db3]
[users_production][0]: allocating [[users_production][0], node[
null], [R], s[UNASSIGNED]] to [[es_db0][4P-*nUocJRTCrYQ5bH3TRUQ][inet[/<
*ip>:9300]]] in order to reuse its unallocated persistent store with
total_size [2.3gb]

I'm seeing these for all the shards though. I also see similar things on
the server with the replica. Is there something I should look for?

Thanks for the help!
Dave

Shay Banon wrote:
Also, any failures in the log?

On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon <kimchy@gmail.com <mailto:
kimchy@gmail.com>> wrote:

Are indexing going on when you execute the test? Can you try and
refresh and see if you still get it?

On Sat, Oct 22, 2011 at 8:46 PM, sakrafd <dave@signalhq.com
mailto:dave@signalhq.com> wrote:
   I'm seeing the same thing:

    curl -X GET
   "http://localhost:9200/users_**production/user/_count?q=user_**
access_id:742http://localhost:9200/users_production/user/_count?q=user_access_id:742
"
{"count":23517808,"_shards":{"total":10,"successful":10,"
failed":0}}
curl -X GET
"http://localhost:9200/users_**production/user/_count?q=user_**
access_id:742http://localhost:9200/users_production/user/_count?q=user_access_id:742
"
{"count":23518180,"_shards":{"total":10,"successful":10,"
failed":0}}
curl -X GET
"http://localhost:9200/users_**production/user/_count?q=user_**
access_id:742http://localhost:9200/users_production/user/_count?q=user_access_id:742
"
{"count":23517808,"_shards":{"total":10,"successful":10,"
failed":0}}
curl -X GET
"http://localhost:9200/users_**production/user/_count?q=user_**
access_id:742http://localhost:9200/users_production/user/_count?q=user_access_id:742
"
{"count":23518180,"_shards":{"total":10,"successful":10,"
failed":0}}
   When I look at doc counts for each shard/replica (through
   elastic-search-head), i can see that one shard and replica is off:

   Primary:
   docs: {
      num_docs: 3082198
      max_doc: 3545713
      deleted_docs: 463515
   }

   Replica:
   docs: {
      num_docs: 3081671
      max_doc: 3520572
      deleted_docs: 438901
   }

   Is there a way to synch these without re-indexing?

   Also, just curious - what's the difference between num_docs
   and max_docs?

   Thanks,
   Dave

   --
   View this message in context:
   http://elasticsearch-users.**115913.n3.nabble.com/Question-**
Inconsistent-count-results-**tp873957p3444017.htmlhttp://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
Sent from the Elasticsearch Users mailing list archive at
Nabble.com.

sakrafd · October 26, 2011, 7:56pm

Thanks! I didn't see any other types of failures - just ones like i
referenced earlier. I ended up tracking down the missing docs and
re-indexing them.

On 10/24/11 5:40 PM, Shay Banon wrote:

You can restart the master node, thats fine, another one will be
elected. Are there any other failures that happened?

On Sun, Oct 23, 2011 at 7:36 PM, dave farkas <dave@signalhq.com
mailto:dave@signalhq.com> wrote:

Indexing is stopped and I see the same results after a refresh -
the replica is off by 362 docs.

The shard that is off - shard 0 - is on the master node. In those
logs I see some recovery/rebalancing statements:

[2011-10-18 19:42:53,551][DEBUG][gateway.local            ]
[es_db3] [users_production][0]: allocating [[users_production][0],
node[null], [P], s[UNASSIGNED]] to
[[es_db3][3OIKkft6QeSGsjRE9TuIKg][inet[/<ip>:9300]]] on primary
allocation

[2011-10-18 19:43:31,655][DEBUG][action.admin.indices.status]
[es_db3] [users_production][0], node[3OIKkft6QeSGsjRE9TuIKg], [P],
s[INITIALIZING]: Failed to execute
[org.elasticsearch.action.admin.indices.status.IndicesStatusRequest@5cb27de5]
org.elasticsearch.index.IndexShardMissingException:
[users_production][0] missing
      at
org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:174)
      at
org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:135)
      at
org.elasticsearch.action.admin.indices.status.TransportIndicesStatusAction.shardOperation(TransportIndicesStatusAction.java:58)
      at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:231)
      at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction.performOperation(TransportBroadcastOperationAction.java:209)
      at
org.elasticsearch.action.support.broadcast.TransportBroadcastOperationAction$AsyncBroadcastAction$1.run(TransportBroadcastOperationAction.java:185)
      at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:636)

[2011-10-18 19:44:02,225][DEBUG][index.gateway            ]
[es_db3] [users_production][0] starting recovery from local ...
[2011-10-18 19:44:05,655][DEBUG][index.gateway            ]
[es_db3] [users_production][0] recovery completed from local, took
[3.4s]
  index    : files           [241] with total_size [2.3gb], took[31ms]
           : recovered_files [0] with total_size [0b]
           : reusing_files   [241] with total_size [2.3gb]
  translog : number_of_operations [0], took [3.4s]
[2011-10-18 19:45:37,834][DEBUG][gateway.local            ]
[es_db3] [users_production][0]: allocating [[users_production][0],
node[
null], [R], s[UNASSIGNED]] to
[[es_db0][4P-nUocJRTCrYQ5bH3TRUQ][inet[/<ip>:9300]]] in order to
reuse its unallocated persistent store with total_size [2.3gb]

I'm seeing these for all the shards though. I also see similar
things on the server with the replica. Is there something I should
look for?

Thanks for the help!
Dave


Shay Banon wrote:

    Also, any failures in the log?

    On Sun, Oct 23, 2011 at 2:17 AM, Shay Banon <kimchy@gmail.com
    <mailto:kimchy@gmail.com> <mailto:kimchy@gmail.com
    <mailto:kimchy@gmail.com>>> wrote:

       Are indexing going on when you execute the test? Can you
    try and
       refresh and see if you still get it?


       On Sat, Oct 22, 2011 at 8:46 PM, sakrafd <dave@signalhq.com
    <mailto:dave@signalhq.com>
    <mailto:dave@signalhq.com <mailto:dave@signalhq.com>>> wrote:

           I'm seeing the same thing:

            curl -X GET
         
     "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
         
     {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
            curl -X GET
         
     "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
         
     {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}
            curl -X GET
         
     "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
         
     {"count":23517808,"_shards":{"total":10,"successful":10,"failed":0}}
            curl -X GET
         
     "http://localhost:9200/users_production/user/_count?q=user_access_id:742"
         
     {"count":23518180,"_shards":{"total":10,"successful":10,"failed":0}}

           When I look at doc counts for each shard/replica (through
           elastic-search-head), i can see that one shard and
    replica is off:

           Primary:
           docs: {
              num_docs: 3082198
              max_doc: 3545713
              deleted_docs: 463515
           }

           Replica:
           docs: {
              num_docs: 3081671
              max_doc: 3520572
              deleted_docs: 438901
           }

           Is there a way to synch these without re-indexing?

           Also, just curious - what's the difference between num_docs
           and max_docs?

           Thanks,
           Dave

           --
           View this message in context:
    http://elasticsearch-users.115913.n3.nabble.com/Question-Inconsistent-count-results-tp873957p3444017.html
           Sent from the ElasticSearch Users mailing list archive at
           Nabble.com.

Topic		Replies	Views
Inconsistent results (again) Elasticsearch	6	287	July 6, 2017
Different counts on match_all Elasticsearch	2	299	July 6, 2017
Different results with/without preference=_primary_first/_replica_first using count API Elasticsearch	1	427	July 6, 2017
Inconsistent search results Elasticsearch	8	3812	July 6, 2017
Random hit counts in every time Elasticsearch	5	864	July 31, 2017

Question: Inconsistent count results

Related topics