# Update Timing **URL:** \ **Category:** Elasticsearch\ **Created:** [August 8, 2013, 7:32pm UTC](https://discuss.elastic.co/t/update-timing/13134 "2013-08-08T19:32:42Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Brandon\_Hilkert\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert_2/32/2182_2.png) [@Brandon\_Hilkert\_2](https://discuss.elastic.co/u/Brandon_Hilkert_2)\ **Post date:** [August 8, 2013, 7:32pm UTC](https://discuss.elastic.co/t/update-timing/13134/1 "2013-08-08T19:32:42Z")
We're setting up an ES cluster and have a question about the sequence of events. The situation I'm attempting to prevent is a little complicated and I just want to make sure updates will be received in the proper order. My proposed setup is 2 ES servers behind a EC2 load balancer. I have clients that reading which is pretty straight forward. However, I have a service that is responsible for making updates to the index. That service is a multithreaded process that's receiving jobs and sending them through the Tire Ruby gem. Let's say I have the following: Update 2 ===\> Update 1 ===\> Index service ===\> Load balancer ====\> ES 1 ``` \=====> ES 2 ``` If I have an index named "orders" and the update pertains to a record that's primary on ES 2. If Update 1 hits the load balancer and it directs the update to ES1, ES1 then has to route the update to ES2. However, update 2 is not far behind and hits the load balancer and goes straight to ES2. I'm worried that update 2 will make it to ES 2 sooner than update 1 because update 1 has to go through that extra hop from ES1 to ultimately get to ES2 where the record lives. Am I worrying about something that is guarded against in ES? Thanks, Brandon -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 8, 2013, 7:37pm UTC](https://discuss.elastic.co/t/update-timing/13134/2 "2013-08-08T19:37:24Z")
Check if versioning with optimistic concurrency control can help you. > **[Versioning | Elastic](https://www.elastic.co/blog/versioning)** > > Upcoming elasticsearch 0.15 has some very cool features baked into it, one of those is versioning. Each document indexed in elasticsearch is now versioned and this allow for some cool operations done ... Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Brandon\_Hilkert\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert_2/32/2182_2.png) [@Brandon\_Hilkert\_2](https://discuss.elastic.co/u/Brandon_Hilkert_2)\ **Post date:** [August 8, 2013, 7:42pm UTC](https://discuss.elastic.co/t/update-timing/13134/3 "2013-08-08T19:42:37Z")
I don't think it helps me necessarily. It's good to know about. But I'm hoping that what I describe is protected against within the ES cluster and that's what I'm more curious about. Thanks for the link though. Brandon On Thu, Aug 8, 2013 at 3:37 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com) > wrote: > Check if versioning with optimistic concurrency control can help you. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/versioning/) > > Jörg > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/XBpPBo-aJ60/unsubscribe](https://groups.google.com/d/topic/elasticsearch/XBpPBo-aJ60/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![mattweber](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mattweber/32/44940_2.png) [@mattweber](https://discuss.elastic.co/u/mattweber)\ **Post date:** [August 8, 2013, 7:58pm UTC](https://discuss.elastic.co/t/update-timing/13134/4 "2013-08-08T19:58:58Z")
Why not replace the load balancer with an ES node that basically becomes the "coordinator" of your cluster? To do this you would just set node.data to false and node.master to true in elasticsearch.yml. This node will only be responsible for sending your "updates" to the correct node, ES1 or ES2. Thanks, Matt Weber On Thu, Aug 8, 2013 at 12:42 PM, Brandon Hilkert \< [brandon@pipelinedealsco.com](mailto:brandon@pipelinedealsco.com)\> wrote: > I don't think it helps me necessarily. It's good to know about. But I'm > hoping that what I describe is protected against within the ES cluster and > that's what I'm more curious about. > > Thanks for the link though. > > Brandon > > On Thu, Aug 8, 2013 at 3:37 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > Check if versioning with optimistic concurrency control can help you. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/blog/versioning/) > > > > Jörg > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/XBpPBo-aJ60/unsubscribe](https://groups.google.com/d/topic/elasticsearch/XBpPBo-aJ60/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 8, 2013, 8:11pm UTC](https://discuss.elastic.co/t/update-timing/13134/5 "2013-08-08T20:11:28Z")
What kind of protection do you have in mind? It is up to the client to create the versioning information for ordering document actions and to take action if it fails. Your updates are traveling independently through the cluster to the location where they persist. Depending on the network speed and CPU load of the nodes, the updates may arrive sooner or later at the shards. It's quite common that some nodes are heavily busy while others are not. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Brandon\_Hilkert\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert_2/32/2182_2.png) [@Brandon\_Hilkert\_2](https://discuss.elastic.co/u/Brandon_Hilkert_2)\ **Post date:** [August 8, 2013, 8:16pm UTC](https://discuss.elastic.co/t/update-timing/13134/6 "2013-08-08T20:16:40Z")
I didn't have anything in particular in mind, but was hoping there was some kind of internal communication that perhaps timestamped something when it got to the routing node so that once it does make it to the correct data node, a comparison could be done to decide whether it's older than data that came straight to the data node. That's just me thinking out loud. Probably a naive approach given the complexity of the cluster. Sounds like there's not anything like this and it's up to me to preserve ordering. I'm still a little unsure how i'd go about this versioning when I just have updates coming from a DB that know nothing about the version. Sounds like i'd have to store some kind of state for each record on the service that is doing the indexing right? On Thursday, August 8, 2013 4:11:28 PM UTC-4, Jörg Prante wrote: > What kind of protection do you have in mind? > > It is up to the client to create the versioning information for ordering > document actions and to take action if it fails. > > Your updates are traveling independently through the cluster to the > location where they persist. Depending on the network speed and CPU load of > the nodes, the updates may arrive sooner or later at the shards. It's quite > common that some nodes are heavily busy while others are not. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 8, 2013, 9:28pm UTC](https://discuss.elastic.co/t/update-timing/13134/7 "2013-08-08T21:28:38Z")
If you can timestamp your data, you can easily bring in your timestamp for versioning, or another incremental counter. It's called "external" versioning. More details are explained here > **[Elasticsearch Versioning Support | Elastic](https://www.elastic.co/blog/elasticsearch-versioning-support)** > > Elasticsearch Versioning Support One of the key principles behind Elasticsearch is to allow you to make the most out of your data. Historically, search was a read-only enterprise where a search engine... Quote: * * * ## With version\_type set to external, Elasticsearch will store the version number as given and will not increment it. Also, instead of checking for an exact match, Elasticsearch will only return a version collision error if the version currently stored is greater or equal to the one in the indexing command. This effectively means “only store this information if no one else has supplied the same or a more recent version in the meantime”. Cluster nodes in ES are not synchronized, they do not communicate about the age of the actions to be performed. Each node is working alone for itself - this optimistic (or byzantine, whatever you like) behavior is one secret of ES extraordinary scalability and performance. No distributed transactions, no waits. If something fails, a recovery (and conflict resolution) is required. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:22am UTC](https://discuss.elastic.co/t/update-timing/13134/8 "2017-07-06T02:22:00Z")