# Is there a way to force a couchdb river update? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 27, 2012, 1:39pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143 "2012-03-27T13:39:27Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Andrius\_Juozapaitis](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@Andrius\_Juozapaitis](https://discuss.elastic.co/u/Andrius_Juozapaitis)\ **Post date:** [March 27, 2012, 1:39pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/1 "2012-03-27T13:39:27Z")
Hey all, Looking for advice here. I am using couchdb as my data store, with elasticsearch handling pretty much all the read operations. I've encountered an issue with the near-real-time indexing through couchdb river though: 1. My spring MVC controller extracts data (say, a product list) using elasticsearch java api 2. The user submits a new product 3. The product is saved in couchdb 4. The controller, following a redirect-after-post pattern, does an http redirect to the product list (see 1.) The problem here is that the controller in step 1. is querying elasticsearch data which hasn't been updated yet by the couchdb river. I am aware there are a number of potential solutions to this problem (also storing the saved data in session, querying couchdb directly, delaying the redirect, using ajax, etc). The most transparent way, IMHO, would be to issue a request to elasticsearch (or couchdb river?) that would block until the couchdb river processes all the pending changes in the \_changes feed, and only then perform the http redirect in the step 4. Is it possible to do through the existing APIs? Are there any other options I missed? In other words, I'd be happy for any constructive feedback here. Best regards, Andrius ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 27, 2012, 6:53pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/2 "2012-03-27T18:53:08Z")
There isn't a way to wait till it gets indexed in elasticsearch, I mean, the river does not expose it. I can't think of a really nice solution except maybe to do a get for hte doc in elasticsearch (without the \_source, i.e. empty fields), and then poll the doc with another get till the \_version changes... On Tue, Mar 27, 2012 at 3:39 PM, Andrius Juozapaitis [andriusj@gmail.com](mailto:andriusj@gmail.com)wrote: > Hey all, > > Looking for advice here. I am using couchdb as my data store, with > elasticsearch handling pretty much all the read operations. I've > encountered an issue with the near-real-time indexing through couchdb > river though: > > 1. My spring MVC controller extracts data (say, a product list) using > elasticsearch java api > 2. The user submits a new product > 3. The product is saved in couchdb > 4. The controller, following a redirect-after-post pattern, does an > http redirect to the product list (see 1.) > > The problem here is that the controller in step 1. is querying > elasticsearch data which hasn't been updated yet by the couchdb river. > I am aware there are a number of potential solutions to this problem > (also storing the saved data in session, querying couchdb directly, > delaying the redirect, using ajax, etc). > > The most transparent way, IMHO, would be to issue a request to > elasticsearch (or couchdb river?) that would block until the couchdb > river processes all the pending changes in the \_changes feed, and only > then perform the http redirect in the step 4. Is it possible to do > through the existing APIs? Are there any other options I missed? In > other words, I'd be happy for any constructive feedback here. > > Best regards, > Andrius ---
**Author:** ![Andrius\_Juozapaitis](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@Andrius\_Juozapaitis](https://discuss.elastic.co/u/Andrius_Juozapaitis)\ **Post date:** [March 28, 2012, 12:50pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/3 "2012-03-28T12:50:59Z")
Looking at the couchdb river code, it looks like it's fairly simple to expose the river state, simply add a flag to CouchdbRiver.Indexer.run() loop: ```auto idle = true; logger.info("finished indexing"); String s; try { s = stream.take(); logger.info("indexing..."); idle = false; ``` Since the stream is a blocking queue, idle flag in CouchdbRiver instance would only be true if there are no pending changes in the feed. The next question is, is there an easy way to expose additional river state through the existing JSON API? \_meta information, perhaps? thanks in advance, Andrius On Tuesday, March 27, 2012 9:53:08 PM UTC+3, kimchy wrote: > There isn't a way to wait till it gets indexed in elasticsearch, I mean, > the river does not expose it. I can't think of a really nice solution > except maybe to do a get for hte doc in elasticsearch (without the \_source, > i.e. empty fields), and then poll the doc with another get till the > \_version changes... > > On Tue, Mar 27, 2012 at 3:39 PM, Andrius Juozapaitis [andriusj@gmail.com](mailto:andriusj@gmail.com)wrote: > > > Hey all, > > > > Looking for advice here. I am using couchdb as my data store, with > > elasticsearch handling pretty much all the read operations. I've > > encountered an issue with the near-real-time indexing through couchdb > > river though: > > > > 1. My spring MVC controller extracts data (say, a product list) using > > elasticsearch java api > > 2. The user submits a new product > > 3. The product is saved in couchdb > > 4. The controller, following a redirect-after-post pattern, does an > > http redirect to the product list (see 1.) > > > > The problem here is that the controller in step 1. is querying > > elasticsearch data which hasn't been updated yet by the couchdb river. > > I am aware there are a number of potential solutions to this problem > > (also storing the saved data in session, querying couchdb directly, > > delaying the redirect, using ajax, etc). > > > > The most transparent way, IMHO, would be to issue a request to > > elasticsearch (or couchdb river?) that would block until the couchdb > > river processes all the pending changes in the \_changes feed, and only > > then perform the http redirect in the step 4. Is it possible to do > > through the existing APIs? Are there any other options I missed? In > > other words, I'd be happy for any constructive feedback here. > > > > Best regards, > > Andrius ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 29, 2012, 12:01pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/4 "2012-03-29T12:01:37Z")
We can add the flag, or update the a "status" json so you can read, but its not enough, since you can't really tell anything if the relevant document you are interested in was processed or not. On Wed, Mar 28, 2012 at 2:50 PM, Andrius Juozapaitis [andriusj@gmail.com](mailto:andriusj@gmail.com)wrote: > Looking at the couchdb river code, it looks like it's fairly simple to > expose the river state, simply add a flag to CouchdbRiver.Indexer.run() > loop: > > ```auto > idle = true; > > logger.info("finished indexing"); > > String s; > > try { > > s = stream.take(); > > logger.info("indexing..."); > > idle = false; > ``` > > Since the stream is a blocking queue, idle flag in CouchdbRiver > instance would only be true if there are no pending changes in the feed. > The next question is, is there an easy way to expose additional river state > through the existing JSON API? \_meta information, perhaps? > > thanks in advance, > Andrius > > On Tuesday, March 27, 2012 9:53:08 PM UTC+3, kimchy wrote: > > > There isn't a way to wait till it gets indexed in elasticsearch, I mean, > > the river does not expose it. I can't think of a really nice solution > > except maybe to do a get for hte doc in elasticsearch (without the \_source, > > i.e. empty fields), and then poll the doc with another get till the > > \_version changes... > > > > On Tue, Mar 27, 2012 at 3:39 PM, Andrius Juozapaitis [andriusj@gmail.com](mailto:andriusj@gmail.com)wrote: > > > > > Hey all, > > > > > > Looking for advice here. I am using couchdb as my data store, with > > > elasticsearch handling pretty much all the read operations. I've > > > encountered an issue with the near-real-time indexing through couchdb > > > river though: > > > > > > 1. My spring MVC controller extracts data (say, a product list) using > > > elasticsearch java api > > > 2. The user submits a new product > > > 3. The product is saved in couchdb > > > 4. The controller, following a redirect-after-post pattern, does an > > > http redirect to the product list (see 1.) > > > > > > The problem here is that the controller in step 1. is querying > > > elasticsearch data which hasn't been updated yet by the couchdb river. > > > I am aware there are a number of potential solutions to this problem > > > (also storing the saved data in session, querying couchdb directly, > > > delaying the redirect, using ajax, etc). > > > > > > The most transparent way, IMHO, would be to issue a request to > > > elasticsearch (or couchdb river?) that would block until the couchdb > > > river processes all the pending changes in the \_changes feed, and only > > > then perform the http redirect in the step 4. Is it possible to do > > > through the existing APIs? Are there any other options I missed? In > > > other words, I'd be happy for any constructive feedback here. > > > > > > Best regards, > > > Andrius ---
**Author:** ![Andrius\_Juozapaitis](https://avatars.discourse-cdn.com/v4/letter/a/e9c0ed/32.png) [@Andrius\_Juozapaitis](https://discuss.elastic.co/u/Andrius_Juozapaitis)\ **Post date:** [March 29, 2012, 12:12pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/5 "2012-03-29T12:12:54Z")
Yes, it's by no means a one-size-fits-all solution. Then again, I gave your initial suggestion another thought - whenever I do an insert/update/delete in my service layer, I get back the doc id+revision. So the most straightforward way would be to just poll ES until that query (id+revision) will return a hit, then do a redirect and be done with it. Not perfect, but easy to implement, and will work just fine in my case. Thanks! Andrius On Thursday, March 29, 2012 3:01:37 PM UTC+3, kimchy wrote: > We can add the flag, or update the a "status" json so you can read, but > its not enough, since you can't really tell anything if the relevant > document you are interested in was processed or not. > > On Wed, Mar 28, 2012 at 2:50 PM, Andrius Juozapaitis [andriusj@gmail.com](mailto:andriusj@gmail.com)wrote: > > > Looking at the couchdb river code, it looks like it's fairly simple to > > expose the river state, simply add a flag to CouchdbRiver.Indexer.run() > > loop: > > > > ```auto > > idle = true; > > >> logger.info("finished indexing"); > > >> String s; > > >> try { > > >> s = stream.take(); > > >> logger.info("indexing..."); > > >> idle = false; > > ``` > > > > Since the stream is a blocking queue, idle flag in CouchdbRiver > > instance would only be true if there are no pending changes in the feed. > > The next question is, is there an easy way to expose additional river state > > through the existing JSON API? \_meta information, perhaps? > > > > thanks in advance, > > Andrius > > > > On Tuesday, March 27, 2012 9:53:08 PM UTC+3, kimchy wrote: > > > > > There isn't a way to wait till it gets indexed in elasticsearch, I mean, > > > the river does not expose it. I can't think of a really nice solution > > > except maybe to do a get for hte doc in elasticsearch (without the \_source, > > > i.e. empty fields), and then poll the doc with another get till the > > > \_version changes... > > > > > > On Tue, Mar 27, 2012 at 3:39 PM, Andrius Juozapaitis \<[andriusj@gmail.com](mailto:andriusj@gmail.com) > > > > > > > wrote: > > > > > > > Hey all, > > > > > > > > Looking for advice here. I am using couchdb as my data store, with > > > > elasticsearch handling pretty much all the read operations. I've > > > > encountered an issue with the near-real-time indexing through couchdb > > > > river though: > > > > > > > > 1. My spring MVC controller extracts data (say, a product list) using > > > > elasticsearch java api > > > > 2. The user submits a new product > > > > 3. The product is saved in couchdb > > > > 4. The controller, following a redirect-after-post pattern, does an > > > > http redirect to the product list (see 1.) > > > > > > > > The problem here is that the controller in step 1. is querying > > > > elasticsearch data which hasn't been updated yet by the couchdb river. > > > > I am aware there are a number of potential solutions to this problem > > > > (also storing the saved data in session, querying couchdb directly, > > > > delaying the redirect, using ajax, etc). > > > > > > > > The most transparent way, IMHO, would be to issue a request to > > > > elasticsearch (or couchdb river?) that would block until the couchdb > > > > river processes all the pending changes in the \_changes feed, and only > > > > then perform the http redirect in the step 4. Is it possible to do > > > > through the existing APIs? Are there any other options I missed? In > > > > other words, I'd be happy for any constructive feedback here. > > > > > > > > Best regards, > > > > Andrius ---
**Author:** ![Alexander\_Petrichkov](https://avatars.discourse-cdn.com/v4/letter/a/a8b319/32.png) [@Alexander\_Petrichkov](https://discuss.elastic.co/u/Alexander_Petrichkov)\ **Post date:** [April 10, 2013, 9:20pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/6 "2013-04-10T21:20:51Z")
Solution: put/delete doc to db and put/delete doc to ES. вторник, 27 марта 2012 г., 16:39:27 UTC+3 пользователь Andrius Juozapaitis написал: > Hey all, > > Looking for advice here. I am using couchdb as my data store, with > elasticsearch handling pretty much all the read operations. I've > encountered an issue with the near-real-time indexing through couchdb > river though: > > 1. My spring MVC controller extracts data (say, a product list) using > elasticsearch java api > 2. The user submits a new product > 3. The product is saved in couchdb > 4. The controller, following a redirect-after-post pattern, does an > http redirect to the product list (see 1.) > > The problem here is that the controller in step 1. is querying > elasticsearch data which hasn't been updated yet by the couchdb river. > I am aware there are a number of potential solutions to this problem > (also storing the saved data in session, querying couchdb directly, > delaying the redirect, using ajax, etc). > > The most transparent way, IMHO, would be to issue a request to > elasticsearch (or couchdb river?) that would block until the couchdb > river processes all the pending changes in the \_changes feed, and only > then perform the http redirect in the step 4. Is it possible to do > through the existing APIs? Are there any other options I missed? In > other words, I'd be happy for any constructive feedback here. > > Best regards, > Andrius -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Alexander\_Petrichkov](https://avatars.discourse-cdn.com/v4/letter/a/a8b319/32.png) [@Alexander\_Petrichkov](https://discuss.elastic.co/u/Alexander_Petrichkov)\ **Post date:** [April 27, 2013, 2:52pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/7 "2013-04-27T14:52:18Z")
After our tests we can see that just putting/deleting document to elastic search index doesn't work. We delete some object from the list (delete from db and delete from elastic search by delete API using http request). Then we make redirect to the list page and deleted object is still there. Think that http request is finishing before index realy will be updated for performance reason. вторник, 27 марта 2012 г., 16:39:27 UTC+3 пользователь Andrius Juozapaitis написал: > Hey all, > > Looking for advice here. I am using couchdb as my data store, with > elasticsearch handling pretty much all the read operations. I've > encountered an issue with the near-real-time indexing through couchdb > river though: > > 1. My spring MVC controller extracts data (say, a product list) using > elasticsearch java api > 2. The user submits a new product > 3. The product is saved in couchdb > 4. The controller, following a redirect-after-post pattern, does an > http redirect to the product list (see 1.) > > The problem here is that the controller in step 1. is querying > elasticsearch data which hasn't been updated yet by the couchdb river. > I am aware there are a number of potential solutions to this problem > (also storing the saved data in session, querying couchdb directly, > delaying the redirect, using ajax, etc). > > The most transparent way, IMHO, would be to issue a request to > elasticsearch (or couchdb river?) that would block until the couchdb > river processes all the pending changes in the \_changes feed, and only > then perform the http redirect in the step 4. Is it possible to do > through the existing APIs? Are there any other options I missed? In > other words, I'd be happy for any constructive feedback here. > > Best regards, > Andrius -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Alexander\_Petrichkov](https://avatars.discourse-cdn.com/v4/letter/a/a8b319/32.png) [@Alexander\_Petrichkov](https://discuss.elastic.co/u/Alexander_Petrichkov)\ **Post date:** [April 27, 2013, 3:25pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/8 "2013-04-27T15:25:03Z")
Look at CouchDB Document API (think we need the same implemented in Elasticsearch for create, update and delete logic): Update Existing Document Request Method: PUT Request URI: /[db\_name]/[doc\_id] Request Headers: _X-Couch-Full-Commit_: true (optional). Ensure that the document has synced to disk before returning success. Request Body: The document itself as a JSON object. It must include the \_rev property, with the revision number of the document the update is based on as the value. Request Parameters: None Description: Updates an existing document and replaces it with a new revision Sample Request URI: [http://127.0.0.1:5984/employees/126](http://127.0.0.1:5984/employees/126) The following is a sample response: {"ok":true,"id":126","rev":"2-4058198378"} вторник, 27 марта 2012 г., 16:39:27 UTC+3 пользователь Andrius Juozapaitis написал: > Hey all, > > Looking for advice here. I am using couchdb as my data store, with > elasticsearch handling pretty much all the read operations. I've > encountered an issue with the near-real-time indexing through couchdb > river though: > > 1. My spring MVC controller extracts data (say, a product list) using > elasticsearch java api > 2. The user submits a new product > 3. The product is saved in couchdb > 4. The controller, following a redirect-after-post pattern, does an > http redirect to the product list (see 1.) > > The problem here is that the controller in step 1. is querying > elasticsearch data which hasn't been updated yet by the couchdb river. > I am aware there are a number of potential solutions to this problem > (also storing the saved data in session, querying couchdb directly, > delaying the redirect, using ajax, etc). > > The most transparent way, IMHO, would be to issue a request to > elasticsearch (or couchdb river?) that would block until the couchdb > river processes all the pending changes in the \_changes feed, and only > then perform the http redirect in the step 4. Is it possible to do > through the existing APIs? Are there any other options I missed? In > other words, I'd be happy for any constructive feedback here. > > Best regards, > Andrius -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Alexander\_Petrichkov](https://avatars.discourse-cdn.com/v4/letter/a/a8b319/32.png) [@Alexander\_Petrichkov](https://discuss.elastic.co/u/Alexander_Petrichkov)\ **Post date:** [April 28, 2013, 1:56pm UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/9 "2013-04-28T13:56:50Z")
Yepp. We have done temporary fix in our code while it's not implemented in ES. You can check that index is updated. It takes some time, but tests show that problem is not reproducible now. вторник, 27 марта 2012 г., 16:39:27 UTC+3 пользователь Andrius Juozapaitis написал: > Hey all, > > Looking for advice here. I am using couchdb as my data store, with > elasticsearch handling pretty much all the read operations. I've > encountered an issue with the near-real-time indexing through couchdb > river though: > > 1. My spring MVC controller extracts data (say, a product list) using > elasticsearch java api > 2. The user submits a new product > 3. The product is saved in couchdb > 4. The controller, following a redirect-after-post pattern, does an > http redirect to the product list (see 1.) > > The problem here is that the controller in step 1. is querying > elasticsearch data which hasn't been updated yet by the couchdb river. > I am aware there are a number of potential solutions to this problem > (also storing the saved data in session, querying couchdb directly, > delaying the redirect, using ajax, etc). > > The most transparent way, IMHO, would be to issue a request to > elasticsearch (or couchdb river?) that would block until the couchdb > river processes all the pending changes in the \_changes feed, and only > then perform the http redirect in the step 4. Is it possible to do > through the existing APIs? Are there any other options I missed? In > other words, I'd be happy for any constructive feedback here. > > Best regards, > Andrius -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:39am UTC](https://discuss.elastic.co/t/is-there-a-way-to-force-a-couchdb-river-update/7143/10 "2017-07-06T02:39:18Z")