Why docCount differs on primary and replicas of the same shard

I'm trying to figure out why executing the same query multiple times gives different results when there's no background indexing involved. What I've found so far (using "explain: true") is that docCount returns different values for primary and replicas of the same shard, where I'd expect that replica is a bit to bit copy of primary shard.

example: first search returns following data (just first

 "_shard": "[unittypes][0]",
 "_node": "-o4ti0RvSZePKozz5_cxqA",

 ...
	{
		"value": 11325,
		"description": "docCount",
		"details": []
	}

second attempt returns results in slightly different score results (even for the same document)

"_shard": "[unittypes][0]",
"_node": "Hgn_AK63Th65mDr9S5-ReQ"
...	
	{
		"value": 11317,
		"description": "docCount",
		"details": []
	}

I've checked shards stats and it seems at least here doc counts are OK.

unittypes 1 p STARTED 233204 124.9mb 10.0.0.5 estest0
unittypes 1 r STARTED 233204 125.1mb 10.0.0.6 estest1
unittypes 2 r STARTED  92044  76.2mb 10.0.0.4 estest2
unittypes 2 p STARTED  92044  77.3mb 10.0.0.6 estest1
unittypes 4 r STARTED      0    261b 10.0.0.4 estest2
unittypes 4 p STARTED      0    261b 10.0.0.5 estest0
unittypes 3 p STARTED      0    261b 10.0.0.4 estest2
unittypes 3 r STARTED      0    261b 10.0.0.5 estest0
unittypes 0 p STARTED 232252 135.9mb 10.0.0.4 estest2
unittypes 0 r STARTED 232252 135.5mb 10.0.0.6 estest1

What can I do to have stable results for multiple queries?
Currently I'm setting search request preference https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-preference.html but as I'm expected to handle higher loads I'd like to be able to utilize all replicas.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.