# Different results on search with several nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [March 28, 2012, 6:11am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160 "2012-03-28T06:11:31Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Byakuya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/byakuya/32/2938_2.png) [@Byakuya](https://discuss.elastic.co/u/Byakuya)\ **Post date:** [March 28, 2012, 6:11am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/1 "2012-03-28T06:11:31Z")
Hello! Here is our problem. We have ElasticSearch cluster with two nodes (20 shards, 2 replicas). There is an index for CouchDb datasource (approx. 2 mil. documents) via river. When we make the same search request with limitation of first 100 results some times, we receive two different sets of results - one after another in turn. Some result elements are skipped but another are added. They are sorted correctly. Total quantity of results is the same, but some elements differ. When we disable one node and only one works, then set of search results we get is stable. Query example (query is built depending on data, which is received from html form): curl -XGET '[http://192.168.0.248:9200/tenderinfo\_index/\_search](http://192.168.0.248:9200/tenderinfo_index/_search)' -d '{ "sort": { "publishDate.value": "desc" }, "from": 0, "fields": [ "\_id", "orderName.value", ... and other fields we need to retrieve ], "query": { "filtered": { "filter": { "bool": { "must": [ { "match\_all": {} } ] } }, "query": { "bool": { "must": [ { "bool": { "should": [ { "query\_string": { "query": "разработка сайта", "default\_operator": "AND", "default\_field": "orderName.value", "analyzer": "russian" } }, { "nested": { "path": "lots", "score\_mode": "avg", "query": { "query\_string": { "query": "разработка сайта", "default\_operator": "AND", "default\_field": "lots.subject.value", "analyzer": "russian" } } } } ] } } ] } } } }, "size": 100 }' The goal of our task is to get hash on search results to detect new data and perform some actions with it. But with interleaved sets of results hash is different at the same query. How this problem can be solved? We appreciate any suggestions. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 28, 2012, 10:58am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/2 "2012-03-28T10:58:39Z")
It might be that some documents have the same sorting value, and then, when you execute one search and it hits one set of shards, and another which hits another set of shards (copies of the data), you will get different results (but correct sorting). You have the optino to specify a "preference" when searching: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html), specifically, check the "custom string value"). This can ensure two searches will use the same shards. On Wed, Mar 28, 2012 at 8:11 AM, Byakuya [mukhin.vladimir@googlemail.com](mailto:mukhin.vladimir@googlemail.com)wrote: > Hello! > Here is our problem. We have Elasticsearch cluster with two nodes (20 > shards, 2 replicas). There is an index for CouchDb datasource (approx. > 2 mil. documents) via river. When we make the same search request with > limitation of first 100 results some times, we receive two different > sets of results - one after another in turn. Some result elements are > skipped but another are added. They are sorted correctly. Total > quantity of results is the same, but some elements differ. When we > disable one node and only one works, then set of search results we get > is stable. > > Query example (query is built depending on data, which is received > from html form): > > curl -XGET '[http://192.168.0.248:9200/tenderinfo\_index/\_search](http://192.168.0.248:9200/tenderinfo_index/_search)' -d '{ > "sort": { > "publishDate.value": "desc" > }, > "from": 0, > "fields": [ > "\_id", > "orderName.value", > ... and other fields we need to retrieve > ], > "query": { > "filtered": { > "filter": { > "bool": { > "must": [ > { > "match\_all": {} > } > ] > } > }, > "query": { > "bool": { > "must": [ > { > "bool": { > "should": [ > { > "query\_string": { > "query": "разработка > сайта", > "default\_operator": > "AND", > "default\_field": > "orderName.value", > "analyzer": "russian" > } > }, > { > "nested": { > "path": "lots", > "score\_mode": "avg", > "query": { > "query\_string": { > "query": > "разработка сайта", > > "default\_operator": "AND", > "default\_field": > "lots.subject.value", > "analyzer": > "russian" > } > } > } > } > ] > } > } > ] > } > } > } > }, > "size": 100 > }' > > The goal of our task is to get hash on search results to detect new > data and perform some actions with it. But with interleaved sets of > results hash is different at the same query. > How this problem can be solved? We appreciate any suggestions. ---
**Author:** ![Byakuya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/byakuya/32/2938_2.png) [@Byakuya](https://discuss.elastic.co/u/Byakuya)\ **Post date:** [April 3, 2012, 6:22am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/3 "2012-04-03T06:22:12Z")
Thank you very much! Worked as a charm. On 28 мар, 14:58, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > It might be that some documents have the same sorting value, and then, when > you execute one search and it hits one set of shards, and another which > hits another set of shards (copies of the data), you will get different > results (but correct sorting). > > You have the optino to specify a "preference" when searching:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html), > specifically, check the "custom string value"). This can ensure two > searches will use the same shards. ---
**Author:** ![Shaun](https://avatars.discourse-cdn.com/v4/letter/s/f07891/32.png) [@Shaun](https://discuss.elastic.co/u/Shaun)\ **Post date:** [April 6, 2012, 9:52am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/4 "2012-04-06T09:52:54Z")
thanks for this answer as I've just been sitting here puzzling over the same thing - click refresh and the hits I get back in first size=25 alternate between two diff 25 sets.. based on your advice I've tried using sessionid in preference but have no idea whether I've done it right, I stuck it in the query DSL: {"query":{"filtered":{"query":{"query\_string": {"query":"ProductionSchool:british","default\_operator":"OR"}},"filter": {"and":[{"exists":{"field":"ObjectNumber"}},{"term": {"Category":"painting"}}]}},"preference":"b8164e5587009099581f89faa5bde211"},"from": 0,"size":"25","facets":{"Dept":{"terms":{"field":"Dept"}},"Category": {"terms":{"field":"Category","size":"50"}},"Name":{"terms": {"field":"Name","size":"50"}},"Material":{"terms": {"field":"Material","size":"50"}},"Technique":{"terms": {"field":"Technique","size":"50"}},"ProductionPlaceName":{"terms": {"field":"ProductionPlaceName","size":"50"}},"RRFlag":{"terms": {"field":"RRFlag"}},"Maker":{"terms": {"field":"Maker.asterm","size":"50"}}}} I have two ES's running and the only config I have done is to name the group and set the master - everything else ES is looking after thanks for any help Shaun On Mar 28, 11:58 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > It might be that some documents have the same sorting value, and then, when > you execute one search and it hits one set of shards, and another which > hits another set of shards (copies of the data), you will get different > results (but correct sorting). > > You have the optino to specify a "preference" when searching:[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html), > specifically, check the "custom string value"). This can ensure two > searches will use the same shards. > > On Wed, Mar 28, 2012 at 8:11 AM, Byakuya [mukhin.vladi...@googlemail.com](mailto:mukhin.vladi...@googlemail.com)wrote: > > > Hello! > > Here is our problem. We have Elasticsearch cluster with two nodes (20 > > shards, 2 replicas). There is an index for CouchDb datasource (approx. > > 2 mil. documents) via river. When we make the same search request with > > limitation of first 100 results some times, we receive two different > > sets of results - one after another in turn. Some result elements are > > skipped but another are added. They are sorted correctly. Total > > quantity of results is the same, but some elements differ. When we > > disable one node and only one works, then set of search results we get > > is stable. > > > Query example (query is built depending on data, which is received > > from html form): > > > curl -XGET '[http://192.168.0.248:9200/tenderinfo\_index/\_search'-d](http://192.168.0.248:9200/tenderinfo_index/_search'-d) '{ > > "sort": { > > "publishDate.value": "desc" > > }, > > "from": 0, > > "fields": [ > > "\_id", > > "orderName.value", > > ... and other fields we need to retrieve > > ], > > "query": { > > "filtered": { > > "filter": { > > "bool": { > > "must": [ > > { > > "match\_all": {} > > } > > ] > > } > > }, > > "query": { > > "bool": { > > "must": [ > > { > > "bool": { > > "should": [ > > { > > "query\_string": { > > "query": "разработка > > сайта", > > "default\_operator": > > "AND", > > "default\_field": > > "orderName.value", > > "analyzer": "russian" > > } > > }, > > { > > "nested": { > > "path": "lots", > > "score\_mode": "avg", > > "query": { > > "query\_string": { > > "query": > > "разработка сайта", > > > "default\_operator": "AND", > > "default\_field": > > "lots.subject.value", > > "analyzer": > > "russian" > > } > > } > > } > > } > > ] > > } > > } > > ] > > } > > } > > } > > }, > > "size": 100 > > }' > > > The goal of our task is to get hash on search results to detect new > > data and perform some actions with it. But with interleaved sets of > > results hash is different at the same query. > > How this problem can be solved? We appreciate any suggestions. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [April 8, 2012, 6:08pm UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/5 "2012-04-08T18:08:28Z")
The preference should be passed as part of the URI request, i.e.: /\_search?preference=xxx, thats because it is required by the node receiving the search request to properly distribute it. On Fri, Apr 6, 2012 at 12:52 PM, Shaun [cybergate9@googlemail.com](mailto:cybergate9@googlemail.com) wrote: > thanks for this answer as I've just been sitting here puzzling over > the same thing - click refresh and the hits I get back in first > size=25 alternate between two diff 25 sets.. > based on your advice I've tried using sessionid in preference but have > no idea whether I've done it right, I stuck it in the query DSL: > > {"query":{"filtered":{"query":{"query\_string": > {"query":"ProductionSchool:british","default\_operator":"OR"}},"filter": > {"and":[{"exists":{"field":"ObjectNumber"}},{"term": > > {"Category":"painting"}}]}},"preference":"b8164e5587009099581f89faa5bde211"},"from": > 0,"size":"25","facets":{"Dept":{"terms":{"field":"Dept"}},"Category": > {"terms":{"field":"Category","size":"50"}},"Name":{"terms": > {"field":"Name","size":"50"}},"Material":{"terms": > {"field":"Material","size":"50"}},"Technique":{"terms": > {"field":"Technique","size":"50"}},"ProductionPlaceName":{"terms": > {"field":"ProductionPlaceName","size":"50"}},"RRFlag":{"terms": > {"field":"RRFlag"}},"Maker":{"terms": > {"field":"Maker.asterm","size":"50"}}}} > > I have two ES's running and the only config I have done is to name the > group and set the master - everything else ES is looking after > > thanks for any help > Shaun > > On Mar 28, 11:58 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > It might be that some documents have the same sorting value, and then, > > when > > you execute one search and it hits one set of shards, and another which > > hits another set of shards (copies of the data), you will get different > > results (but correct sorting). > > > > You have the optino to specify a "preference" when searching: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html), > > specifically, check the "custom string value"). This can ensure two > > searches will use the same shards. > > > > On Wed, Mar 28, 2012 at 8:11 AM, Byakuya \<[mukhin.vladi...@googlemail.com](mailto:mukhin.vladi...@googlemail.com) > > wrote: > > > > > Hello! > > > Here is our problem. We have Elasticsearch cluster with two nodes (20 > > > shards, 2 replicas). There is an index for CouchDb datasource (approx. > > > 2 mil. documents) via river. When we make the same search request with > > > limitation of first 100 results some times, we receive two different > > > sets of results - one after another in turn. Some result elements are > > > skipped but another are added. They are sorted correctly. Total > > > quantity of results is the same, but some elements differ. When we > > > disable one node and only one works, then set of search results we get > > > is stable. > > > > > Query example (query is built depending on data, which is received > > > from html form): > > > > > curl -XGET '[http://192.168.0.248:9200/tenderinfo\_index/\_search'-d](http://192.168.0.248:9200/tenderinfo_index/_search'-d) '{ > > > "sort": { > > > "publishDate.value": "desc" > > > }, > > > "from": 0, > > > "fields": [ > > > "\_id", > > > "orderName.value", > > > ... and other fields we need to retrieve > > > ], > > > "query": { > > > "filtered": { > > > "filter": { > > > "bool": { > > > "must": [ > > > { > > > "match\_all": {} > > > } > > > ] > > > } > > > }, > > > "query": { > > > "bool": { > > > "must": [ > > > { > > > "bool": { > > > "should": [ > > > { > > > "query\_string": { > > > "query": "разработка > > > сайта", > > > "default\_operator": > > > "AND", > > > "default\_field": > > > "orderName.value", > > > "analyzer": "russian" > > > } > > > }, > > > { > > > "nested": { > > > "path": "lots", > > > "score\_mode": "avg", > > > "query": { > > > "query\_string": { > > > "query": > > > "разработка сайта", > > > > > "default\_operator": "AND", > > > "default\_field": > > > "lots.subject.value", > > > "analyzer": > > > "russian" > > > } > > > } > > > } > > > } > > > ] > > > } > > > } > > > ] > > > } > > > } > > > } > > > }, > > > "size": 100 > > > }' > > > > > The goal of our task is to get hash on search results to detect new > > > data and perform some actions with it. But with interleaved sets of > > > results hash is different at the same query. > > > How this problem can be solved? We appreciate any suggestions. ---
**Author:** ![Shaun](https://avatars.discourse-cdn.com/v4/letter/s/f07891/32.png) [@Shaun](https://discuss.elastic.co/u/Shaun)\ **Post date:** [April 9, 2012, 5:33am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/6 "2012-04-09T05:33:32Z")
oh doh me! didn't try the simplest way to do it did I? 🙂 thanks so much (and btw the way for elasticsearch too) it's now driving our new collections explorer at: > **[Explore our collection](https://fitzmuseum.cam.ac.uk/explore-our-collection)** > > A search page for our highlight objects Shaun Osborne On Apr 8, 7:08 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > The preference should be passed as part of the URI request, i.e.: > /\_search?preference=xxx, thats because it is required by the > node receiving the search request to properly distribute it. > > On Fri, Apr 6, 2012 at 12:52 PM, Shaun [cyberga...@googlemail.com](mailto:cyberga...@googlemail.com) wrote: > > > thanks for this answer as I've just been sitting here puzzling over > > the same thing - click refresh and the hits I get back in first > > size=25 alternate between two diff 25 sets.. > > based on your advice I've tried using sessionid in preference but have > > no idea whether I've done it right, I stuck it in the query DSL: > > > {"query":{"filtered":{"query":{"query\_string": > > {"query":"ProductionSchool:british","default\_operator":"OR"}},"filter": > > {"and":[{"exists":{"field":"ObjectNumber"}},{"term": > > > {"Category":"painting"}}]}},"preference":"b8164e5587009099581f89faa5bde211" },"from": > > 0,"size":"25","facets":{"Dept":{"terms":{"field":"Dept"}},"Category": > > {"terms":{"field":"Category","size":"50"}},"Name":{"terms": > > {"field":"Name","size":"50"}},"Material":{"terms": > > {"field":"Material","size":"50"}},"Technique":{"terms": > > {"field":"Technique","size":"50"}},"ProductionPlaceName":{"terms": > > {"field":"ProductionPlaceName","size":"50"}},"RRFlag":{"terms": > > {"field":"RRFlag"}},"Maker":{"terms": > > {"field":"Maker.asterm","size":"50"}}}} > > > I have two ES's running and the only config I have done is to name the > > group and set the master - everything else ES is looking after > > > thanks for any help > > Shaun > > > On Mar 28, 11:58 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > It might be that some documents have the same sorting value, and then, > > > when > > > you execute one search and it hits one set of shards, and another which > > > hits another set of shards (copies of the data), you will get different > > > results (but correct sorting). > > > > You have the optino to specify a "preference" when searching: > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/preference.html), > > > specifically, check the "custom string value"). This can ensure two > > > searches will use the same shards. > > > > On Wed, Mar 28, 2012 at 8:11 AM, Byakuya \<[mukhin.vladi...@googlemail.com](mailto:mukhin.vladi...@googlemail.com) > > > wrote: > > > > > Hello! > > > > Here is our problem. We have Elasticsearch cluster with two nodes (20 > > > > shards, 2 replicas). There is an index for CouchDb datasource (approx. > > > > 2 mil. documents) via river. When we make the same search request with > > > > limitation of first 100 results some times, we receive two different > > > > sets of results - one after another in turn. Some result elements are > > > > skipped but another are added. They are sorted correctly. Total > > > > quantity of results is the same, but some elements differ. When we > > > > disable one node and only one works, then set of search results we get > > > > is stable. > > > > > Query example (query is built depending on data, which is received > > > > from html form): > > > > > curl -XGET '[http://192.168.0.248:9200/tenderinfo\_index/\_search'-d'](http://192.168.0.248:9200/tenderinfo_index/_search'-d'){ > > > > "sort": { > > > > "publishDate.value": "desc" > > > > }, > > > > "from": 0, > > > > "fields": [ > > > > "\_id", > > > > "orderName.value", > > > > ... and other fields we need to retrieve > > > > ], > > > > "query": { > > > > "filtered": { > > > > "filter": { > > > > "bool": { > > > > "must": [ > > > > { > > > > "match\_all": {} > > > > } > > > > ] > > > > } > > > > }, > > > > "query": { > > > > "bool": { > > > > "must": [ > > > > { > > > > "bool": { > > > > "should": [ > > > > { > > > > "query\_string": { > > > > "query": "разработка > > > > сайта", > > > > "default\_operator": > > > > "AND", > > > > "default\_field": > > > > "orderName.value", > > > > "analyzer": "russian" > > > > } > > > > }, > > > > { > > > > "nested": { > > > > "path": "lots", > > > > "score\_mode": "avg", > > > > "query": { > > > > "query\_string": { > > > > "query": > > > > "разработка сайта", > > > > > "default\_operator": "AND", > > > > "default\_field": > > > > "lots.subject.value", > > > > "analyzer": > > > > "russian" > > > > } > > > > } > > > > } > > > > } > > > > ] > > > > } > > > > } > > > > ] > > > > } > > > > } > > > > } > > > > }, > > > > "size": 100 > > > > }' > > > > > The goal of our task is to get hash on search results to detect new > > > > data and perform some actions with it. But with interleaved sets of > > > > results hash is different at the same query. > > > > How this problem can be solved? We appreciate any suggestions. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:33am UTC](https://discuss.elastic.co/t/different-results-on-search-with-several-nodes/7160/7 "2017-07-06T03:33:09Z")