# Get consistency **URL:** \ **Category:** Elasticsearch\ **Created:** [June 3, 2010, 4:11pm UTC](https://discuss.elastic.co/t/get-consistency/2994 "2010-06-03T16:11:42Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Paul\_Loy](https://avatars.discourse-cdn.com/v4/letter/p/ad7895/32.png) [@Paul\_Loy](https://discuss.elastic.co/u/Paul_Loy)\ **Post date:** [June 3, 2010, 4:11pm UTC](https://discuss.elastic.co/t/get-consistency/2994/1 "2010-06-03T16:11:42Z")
Hi Shay, the current model we have for our data allows partial Documents to be indexed. At some point in the future we may then add the missing fields. Currently I do this like so (in pseudo): function update(PartialDoc update) { PartialDoc doc = _get_(); doc.merge(update); _index_(doc); } The problem is that the initial indexing of the doc may not have happened before we call _get_ so after the merge+_index_,\* \*we have lost the previously indexed fields. Is there a way to specify a consistency level in ElasticSearch (like Cassandra), or couls Elastic Search internally provide update functionality? Thanks, Paul ## -- Paul Loy [paul@keteracel.com](mailto:paul@keteracel.com) [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [June 3, 2010, 6:45pm UTC](https://discuss.elastic.co/t/get-consistency/2994/2 "2010-06-03T18:45:49Z")
Hi Paul, It sounds like you have a similar requirement to us. After a brief discussion with Shay, I've created the following issue on github > **[Issues · elastic/elasticsearch](https://github.com/elastic/elasticsearch/issues#issue/204)** > > Free and Open, Distributed, RESTful Search Engine. Contribute to elastic/elasticsearch development by creating an account on GitHub. It is not clear what the best way to handle this requirement is. Having get method work in real-time may be one potential solution. There may be better ways to solve this as well. Regards, Berkay Mollamustafaoglu [http://www.ifountain.com](http://www.ifountain.com) mberkay on yahoo, google and skype On Thu, Jun 3, 2010 at 12:11 PM, Paul Loy [keteracel@gmail.com](mailto:keteracel@gmail.com) wrote: > Hi Shay, > > the current model we have for our data allows partial Documents to be > indexed. At some point in the future we may then add the missing fields. > Currently I do this like so (in pseudo): > > function update(PartialDoc update) { > PartialDoc doc = _get_(); > doc.merge(update); > _index_(doc); > } > > The problem is that the initial indexing of the doc may not have happened > before we call _get_ so after the merge+_index_,\* \*we have lost the > previously indexed fields. > > Is there a way to specify a consistency level in Elasticsearch (like > Cassandra), or couls Elastic Search internally provide update functionality? > > Thanks, > > Paul > > ## -- > > Paul Loy > [paul@keteracel.com](mailto:paul@keteracel.com) > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![talsalmona](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/talsalmona/32/3316_2.png) [@talsalmona](https://discuss.elastic.co/u/talsalmona)\ **Post date:** [June 3, 2010, 7:46pm UTC](https://discuss.elastic.co/t/get-consistency/2994/3 "2010-06-03T19:46:10Z")
Wouldn't a "refresh" of the index right before the search solve this? ([http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/) indices/refresh/) On Jun 3, 9:45 pm, Berkay Mollamustafaoglu [mber...@gmail.com](mailto:mber...@gmail.com) wrote: > Hi Paul, > > It sounds like you have a similar requirement to us. After a brief > discussion with Shay, I've created the following issue on githubhttp://github.com/elasticsearch/elasticsearch/issues#issue/204 > > It is not clear what the best way to handle this requirement is. Having get > method work in real-time may be one potential solution. There may be better > ways to solve this as well. > > Regards, > Berkay Mollamustafaogluhttp://www.ifountain.com > mberkay on yahoo, google and skype > > On Thu, Jun 3, 2010 at 12:11 PM, Paul Loy [ketera...@gmail.com](mailto:ketera...@gmail.com) wrote: > > > Hi Shay, > > > the current model we have for our data allows partial Documents to be > > indexed. At some point in the future we may then add the missing fields. > > Currently I do this like so (in pseudo): > > > function update(PartialDoc update) { > > PartialDoc doc = _get_(); > > doc.merge(update); > > _index_(doc); > > } > > > The problem is that the initial indexing of the doc may not have happened > > before we call _get_ so after the merge+_index_,\* \*we have lost the > > previously indexed fields. > > > Is there a way to specify a consistency level in Elasticsearch (like > > Cassandra), or couls Elastic Search internally provide update functionality? > > > Thanks, > > > Paul > > > ## -- > > > > Paul Loy > > [p...@keteracel.com](mailto:p...@keteracel.com) > > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [June 3, 2010, 7:48pm UTC](https://discuss.elastic.co/t/get-consistency/2994/4 "2010-06-03T19:48:27Z")
Yes, but calling refresh before (or after) ever index operation would have a very high performance penalty, hence not really a feasible solution when performance is a concern. Regards, Berkay Mollamustafaoglu [http://www.ifountain.com](http://www.ifountain.com) mberkay on yahoo, google and skype On Thu, Jun 3, 2010 at 3:46 PM, Tal [talsalmona@gmail.com](mailto:talsalmona@gmail.com) wrote: > Wouldn't a "refresh" of the index right before the search solve this? > ([http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/) > indices/refresh/) > > On Jun 3, 9:45 pm, Berkay Mollamustafaoglu [mber...@gmail.com](mailto:mber...@gmail.com) wrote: > > > Hi Paul, > > > > It sounds like you have a similar requirement to us. After a brief > > discussion with Shay, I've created the following issue on githubhttp:// > > [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues#issue/204) > > > > It is not clear what the best way to handle this requirement is. Having > > get > > method work in real-time may be one potential solution. There may be > > better > > ways to solve this as well. > > > > Regards, > > Berkay Mollamustafaogluhttp://www.ifountain.com > > mberkay on yahoo, google and skype > > > > On Thu, Jun 3, 2010 at 12:11 PM, Paul Loy [ketera...@gmail.com](mailto:ketera...@gmail.com) wrote: > > > > > Hi Shay, > > > > > the current model we have for our data allows partial Documents to be > > > indexed. At some point in the future we may then add the missing > > > fields. > > > Currently I do this like so (in pseudo): > > > > > function update(PartialDoc update) { > > > PartialDoc doc = _get_(); > > > doc.merge(update); > > > _index_(doc); > > > } > > > > > The problem is that the initial indexing of the doc may not have > > > happened > > > before we call _get_ so after the merge+_index_,\* \*we have lost the > > > previously indexed fields. > > > > > Is there a way to specify a consistency level in Elasticsearch (like > > > Cassandra), or couls Elastic Search internally provide update > > > functionality? > > > > > Thanks, > > > > > Paul > > > > > ## -- > > > > > > Paul Loy > > > [p...@keteracel.com](mailto:p...@keteracel.com) > > > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![Paul\_Loy](https://avatars.discourse-cdn.com/v4/letter/p/ad7895/32.png) [@Paul\_Loy](https://discuss.elastic.co/u/Paul_Loy)\ **Post date:** [June 3, 2010, 10:59pm UTC](https://discuss.elastic.co/t/get-consistency/2994/5 "2010-06-03T22:59:12Z")
The problem with refresh and get is that I added a refresh for a demo but it can still break. Also, as Berkay has stated, it would be very costly to refresh an entire index in order to retrieve a single record consistently. Also, what happens if another index request comes in after the refresh but before the get? I think it would be much better to handle updates as a special case inside Elastic Search. So this is a slightly different problem/solution than issue 204 in my opinion. Thanks, Paul. On Thu, Jun 3, 2010 at 8:48 PM, Berkay Mollamustafaoglu [mberkay@gmail.com](mailto:mberkay@gmail.com)wrote: > Yes, but calling refresh before (or after) ever index operation would have > a very high performance penalty, hence not really a feasible solution when > performance is a concern. > > Regards, > Berkay Mollamustafaoglu > > [http://www.ifountain.com](http://www.ifountain.com) > mberkay on yahoo, google and skype > > On Thu, Jun 3, 2010 at 3:46 PM, Tal [talsalmona@gmail.com](mailto:talsalmona@gmail.com) wrote: > > > Wouldn't a "refresh" of the index right before the search solve this? > > ([http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/) > > indices/refresh/[http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/indices/refresh/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/indices/refresh/) > > ) > > > > On Jun 3, 9:45 pm, Berkay Mollamustafaoglu [mber...@gmail.com](mailto:mber...@gmail.com) wrote: > > > > > Hi Paul, > > > > > > It sounds like you have a similar requirement to us. After a brief > > > discussion with Shay, I've created the following issue on githubhttp:// > > > [GitHub · Where software is built](http://github.com/elasticsearch/elasticsearch/issues#issue/204) > > > > > > It is not clear what the best way to handle this requirement is. Having > > > get > > > method work in real-time may be one potential solution. There may be > > > better > > > ways to solve this as well. > > > > > > Regards, > > > Berkay Mollamustafaogluhttp://www.ifountain.com > > > mberkay on yahoo, google and skype > > > > > > On Thu, Jun 3, 2010 at 12:11 PM, Paul Loy [ketera...@gmail.com](mailto:ketera...@gmail.com) wrote: > > > > > > > Hi Shay, > > > > > > > the current model we have for our data allows partial Documents to be > > > > indexed. At some point in the future we may then add the missing > > > > fields. > > > > Currently I do this like so (in pseudo): > > > > > > > function update(PartialDoc update) { > > > > PartialDoc doc = _get_(); > > > > doc.merge(update); > > > > _index_(doc); > > > > } > > > > > > > The problem is that the initial indexing of the doc may not have > > > > happened > > > > before we call _get_ so after the merge+_index_,\* \*we have lost the > > > > previously indexed fields. > > > > > > > Is there a way to specify a consistency level in Elasticsearch (like > > > > Cassandra), or couls Elastic Search internally provide update > > > > functionality? > > > > > > > Thanks, > > > > > > > Paul > > > > > > > ## -- > > > > > > > > Paul Loy > > > > [p...@keteracel.com](mailto:p...@keteracel.com) > > > > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ## -- Paul Loy [paul@keteracel.com](mailto:paul@keteracel.com) [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [June 4, 2010, 4:24pm UTC](https://discuss.elastic.co/t/get-consistency/2994/6 "2010-06-04T16:24:46Z")
Yes, it is. What you want is something similar to optimistic locking (aka etag and "If-Match" in REST). When you do a GET or search, you get back etags with each resource, and when you do an index, you provide it, and if they don't match, then it fails. All are possible, real time gets, etags, and so on. But, I am currently focusing on making elasticsearch the best search solution out there. There are some search features that I am focusing for the next release. -shay.banon On Fri, Jun 4, 2010 at 1:59 AM, Paul Loy [keteracel@gmail.com](mailto:keteracel@gmail.com) wrote: > The problem with refresh and get is that I added a refresh for a demo but > it can still break. Also, as Berkay has stated, it would be very costly to > refresh an entire index in order to retrieve a single record consistently. > > Also, what happens if another index request comes in after the refresh but > before the get? > > I think it would be much better to handle updates as a special case inside > Elastic Search. So this is a slightly different problem/solution than issue > 204 in my opinion. > > Thanks, > > Paul. > > On Thu, Jun 3, 2010 at 8:48 PM, Berkay Mollamustafaoglu \<[mberkay@gmail.com](mailto:mberkay@gmail.com) > > > wrote: > > > Yes, but calling refresh before (or after) ever index operation would have > > a very high performance penalty, hence not really a feasible solution when > > performance is a concern. > > > > Regards, > > Berkay Mollamustafaoglu > > > > [http://www.ifountain.com](http://www.ifountain.com) > > mberkay on yahoo, google and skype > > > > On Thu, Jun 3, 2010 at 3:46 PM, Tal [talsalmona@gmail.com](mailto:talsalmona@gmail.com) wrote: > > > > > Wouldn't a "refresh" of the index right before the search solve this? > > > ([http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/) > > > indices/refresh/[http://www.elasticsearch.com/docs/elasticsearch/rest\_api/admin/indices/refresh/](http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/indices/refresh/) > > > ) > > > > > > On Jun 3, 9:45 pm, Berkay Mollamustafaoglu [mber...@gmail.com](mailto:mber...@gmail.com) wrote: > > > > > > > Hi Paul, > > > > > > > > It sounds like you have a similar requirement to us. After a brief > > > > discussion with Shay, I've created the following issue on githubhttp:// > > > > [Issues · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues#issue/204) > > > > > > > > It is not clear what the best way to handle this requirement is. Having > > > > get > > > > method work in real-time may be one potential solution. There may be > > > > better > > > > ways to solve this as well. > > > > > > > > Regards, > > > > Berkay Mollamustafaogluhttp://www.ifountain.com > > > > mberkay on yahoo, google and skype > > > > > > > > On Thu, Jun 3, 2010 at 12:11 PM, Paul Loy [ketera...@gmail.com](mailto:ketera...@gmail.com) wrote: > > > > > > > > > Hi Shay, > > > > > > > > > the current model we have for our data allows partial Documents to be > > > > > indexed. At some point in the future we may then add the missing > > > > > fields. > > > > > Currently I do this like so (in pseudo): > > > > > > > > > function update(PartialDoc update) { > > > > > PartialDoc doc = _get_(); > > > > > doc.merge(update); > > > > > _index_(doc); > > > > > } > > > > > > > > > The problem is that the initial indexing of the doc may not have > > > > > happened > > > > > before we call _get_ so after the merge+_index_,\* \*we have lost the > > > > > previously indexed fields. > > > > > > > > > Is there a way to specify a consistency level in Elasticsearch (like > > > > > Cassandra), or couls Elastic Search internally provide update > > > > > functionality? > > > > > > > > > Thanks, > > > > > > > > > Paul > > > > > > > > > ## -- > > > > > > > > > > Paul Loy > > > > > [p...@keteracel.com](mailto:p...@keteracel.com) > > > > > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) > > ## -- > > Paul Loy > [paul@keteracel.com](mailto:paul@keteracel.com) > > [http://www.keteracel.com/paul](http://www.keteracel.com/paul) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:23am UTC](https://discuss.elastic.co/t/get-consistency/2994/7 "2017-07-06T04:23:35Z")