# Different results because of replicas **URL:** \ **Category:** Elasticsearch\ **Created:** [September 26, 2019, 2:40pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256 "2019-09-26T14:40:23Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Ewoks](https://avatars.discourse-cdn.com/v4/letter/e/ac8455/32.png) [@Ewoks](https://discuss.elastic.co/u/Ewoks)\ **Post date:** [September 26, 2019, 2:40pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/1 "2019-09-26T14:40:23Z")
Hello, my Elasticsearch 6.x Index has 1 shard and **2 replicas** with ~20k documents. When I make 2 consecutive requests with exactly same parameters, I get same number of hits with slightly differently scored results, i.e.: - max\_score is different (5.1018233 vs 5.106115) - one specific document is on the top of the results when request is sent once, but same document is scored so it is fourth in the result list after other request Every time I made request, result will bounce between one of these 2 variants. For testing purposes, I added _"preference=\_primary"_ & _"preference=\_replica"_ to the endpoint and noticed that I get same two different results, depending of preference value. Querying "\_primary" & "\_replica" for number of documents confirmed that both has the same number of documents. Anybody can explain what/why is this happening? Anybody can help me to get consistent results despite having more than one replica? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 26, 2019, 2:52pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/2 "2019-09-26T14:52:47Z")
Housekeeping merge operations happen independently on shards. - The number of deleted documents will therefore vary between shards. - Deleted documents are included in the total number of documents included in the Lucene index that is a shard. - The total number of documents in the Lucene index is used in IDF (inverse document frequency) scoring which means replicas can score the same query on the same docs slightly differently depending on merge activity, This is one of the reasons the `preference` setting can take something like a user session ID as a value to ensure each user returns to the same choice of replica. Of course, ongoing indexing activity and merging can change matters even on the same replica so users shouldn't expect to have a completely stable view. ---
**Author:** ![Ewoks](https://avatars.discourse-cdn.com/v4/letter/e/ac8455/32.png) [@Ewoks](https://discuss.elastic.co/u/Ewoks)\ **Post date:** [September 26, 2019, 3:04pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/3 "2019-09-26T15:04:21Z")
The issue here is that user searches for document by its title and get it on the top of results, but if user search for it again, it might be 4th in the results list !?!? I would not have an issue if scoring is slightly different & reduced/increased for all projects in a same way, but proposed giving users session ID as preference parameter in practise would mean that some users will get correct scoring and some not (they will get worse result list cause document will not be on the top as expected)... or I got it wrong? p.s. I have 1 shard, but 2 replicas. Do replicas works like shards? I assumed housekeeping would happen on primary and then eventually it will be replicated to replica? Was that wrong assumption? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 26, 2019, 3:16pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/4 "2019-09-26T15:16:41Z")
It's the way it is because of a number of necessary trade-offs. > [@Ewoks](#): > > in practise would mean that some users will get correct scoring and some not What's your definition of "correct"? Presumably, that would be stats based on your indexing having no deletes at all (a "force merged" index). Maintaining that isn't practical so we have to work with imperfect stats for fast results. > [@Ewoks](#): > > Was that wrong assumption? Yes. Under ordinary indexing activity JSON documents are replicated not their indexed form (segment files). Each replica does its own indexing of that JSON content into segment files. If you design a system around replicating segment files there's a tension - you'd want segment files to be small to minimise the content transferred over networks but you'd want segment files to be merged into large ones to make searches efficient. If ordering is critical then among user groups maybe they can share the same session ID used for preference routing? That, or the query is redefined to be less sensitive to IDF ranking. ---
**Author:** ![Ewoks](https://avatars.discourse-cdn.com/v4/letter/e/ac8455/32.png) [@Ewoks](https://discuss.elastic.co/u/Ewoks)\ **Post date:** [September 26, 2019, 3:40pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/5 "2019-09-26T15:40:02Z")
> [@Mark\_Harwood](#): > > What's your definition of "correct"? If for given search parameters, Elasticsearch with no replicas was able to score the document to the top of the result list, let's call that "correct" reference. It is the best ES could do with that index configuration and given search parameters. For me is very surprising that because I decided to have additional replica, **sometimes** I don't get that "correct" result anymore (despite knowing that ES could score specific document better with specific index configuration) and instead there is some other results that user would rate as "worse". Feels like one needs to decide between consistency of good results and having more replicas. Sorry if I sound too critical, but I still don't have solution for the issue > [@Mark\_Harwood](#): > > Each replica does its own indexing of that JSON content into segment files Perfect. So, at the end of indexing of that JSON content, segmented files in _\_replica_ are same as in _\_primary_? I really hope indexing is consistent and happening in the same way on _\_primary_ and _\_replica_ ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 26, 2019, 3:57pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/6 "2019-09-26T15:57:21Z")
> [@Ewoks](#): > > If for given search parameters, Elasticsearch with no replicas was able to score the document to the top of the result list, let's call that "correct" reference. "Correct" can vary from second to second if indexing and merging activity is ongoing. Even on a no-replica system the number one result can move due to other document changes. These changes might be completely unrelated to the documents matching your search. That's part of the user experience - they can't assume they have a stable view of the data if changes are happening. > [@Ewoks](#): > > segmented files in _\_replica_ are same as in _\_primary_ ? No. The set of documents should be the same - the way they are physically organised is different. There may be different search loads or other performance concerns which causes background housekeeping merge activity to take longer which means the merge scheduling operations on each replica will diverge over time. As I explained previously, the merging of deletes will change the IDF scores. If your content is **not** undergoing change then a "force merge" operation should make replicas rank the same (and it's only when you expect an index to have no more changes that it's appropriate to pay the heavy cost of a "force merge"). In a live index with ongoing changes it's typically too heavy a cost to maintain an identical view of relevance ranking across all replicas. Relevance scores are always changing anyway. If you assume you wanted relevance ranking and needed that to be stable across a changing index then you'd essentially have to know all possible search terms in advance (`aardvark` to `zit`) and assign each of them a weighted score (so that `aardvark` matches score more highly than `the` matches). You'd then need to write searches that used these pre-determined boosts for each search term. This is impractical, of course. ---
**Author:** ![Ewoks](https://avatars.discourse-cdn.com/v4/letter/e/ac8455/32.png) [@Ewoks](https://discuss.elastic.co/u/Ewoks)\ **Post date:** [September 26, 2019, 8:59pm UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/7 "2019-09-26T20:59:30Z")
Ok, I understand now slightly better. Can you recommend me some resources to learn more about this so I can make better decisions in future? Thanks for all clarifications ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [September 27, 2019, 8:31am UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/8 "2019-09-27T08:31:32Z")
> [@Ewoks](#): > > Can you recommend me some resources to learn more about this so I can make better decisions in future? Here you go: - This [old article](https://www.elastic.co/blog/understanding-query-then-fetch-vs-dfs-query-then-fetch) has some background to the [DFS](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/search-request-body.html#request-body-search-search-type) search mode. That feature is designed to overcome any scoring bias across _different_ shards. However it won't help you solve your problem which is scoring differences between _replicas of the same shard_. - For a detailed discussion on what might be the pros and cons of segment based replication (copying indexed docs not raw docs) then see [this thread](https://discuss.elastic.co/t/a-new-replication-type-physical-replication/157296/3) - As you've already discovered, the `preference` parameter can help route users to the same choice of replica to try and give some stability (but ongoing indexing will carry on changing order). - For a deep dive on the data structures in the indexes see [this talk](https://berlinbuzzwords.de/15/session/algorithms-and-data-structures-power-lucene-and-elasticsearch) - Mike McCandless's [animation of Lucene segment merging]. ([Changing Bits: Visualizing Lucene's segment merges](http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html)) is an interesting insight. All-in-all distributed search on an actively changing dataset is challenging and requires a number of trade-offs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 25, 2019, 8:44am UTC](https://discuss.elastic.co/t/different-results-because-of-replicas/201256/9 "2019-10-25T08:44:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.