# "wait for refresh" command?
**URL:** https://discuss.elastic.co/t/wait-for-refresh-command/4589
**Category:** Elasticsearch
**Created:** [June 9, 2011, 3:40pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589 "2011-06-09T15:40:10Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Curtis\_Caravone](https://discuss.elastic.co/u/Curtis_Caravone)
#### Post date: [June 9, 2011, 3:40pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/1 "2011-06-09T15:40:10Z")
Is there a way to tell when an indexed document has become visible for
searching, short of forcing an index refresh?
Something like a "wait for refresh" option to the indexing commands (server
will not send response message until the doc has been indexed and the next
refresh has happened)
or even just a "wait for next refresh" command?
This could be very useful for enforcing some level of inter-document
consistency.
For example, in a relational parent/child model (child doc -\> parent doc),
one might want to ensure that the parent doc is visible before any child doc
becomes visible.
Curtis
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 9, 2011, 7:21pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/2 "2011-06-09T19:21:58Z")
No, there isn't one, and its problematic to provide? It means that elasticsearch will need to keep track of the indexed documents, so you will be able to tell if it happened after them. Parent child visibility can be ensure by indexing the parent doc before the child doc for example.
On Thursday, June 9, 2011 at 6:40 PM, Curtis Caravone wrote:
> Is there a way to tell when an indexed document has become visible for searching, short of forcing an index refresh?
>
> Something like a "wait for refresh" option to the indexing commands (server will not send response message until the doc has been indexed and the next refresh has happened)
> or even just a "wait for next refresh" command?
>
> This could be very useful for enforcing some level of inter-document consistency.
>
> For example, in a relational parent/child model (child doc -\> parent doc), one might want to ensure that the parent doc is visible before any child doc becomes visible.
>
> Curtis
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [June 9, 2011, 7:27pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/3 "2011-06-09T19:27:47Z")
I'm not aware of such a feature although it would be really nice to
have such a listener capability.
Maybe you create an issue for this?
Possible workarounds are do refresh yourself (not good 🙂 !) or wait
at least as long the the refresh\_interval is (or two times of this)
Regards,
Peter.
---
### Author:  [@Karussell1](https://discuss.elastic.co/u/Karussell1)
#### Post date: [June 9, 2011, 8:22pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/4 "2011-06-09T20:22:16Z")
On Jun 9, 9:21 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> No, there isn't one, and its problematic to provide? It means that elasticsearch will need to keep track of the indexed documents, so you will be able to tell if it happened after them. Parent child visibility can be ensure by indexing the parent doc before the child doc for example.
E.g. FAST ESP provides listeners for this purposes and especially with
the NRT feature one now cannot give a guarantuee when a doc gets
indexed.
> It means that elasticsearch will need to keep track of the indexed documents
maybe only for a subset? or one can mark only one document as a
trigger?
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 9, 2011, 8:25pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/5 "2011-06-09T20:25:29Z")
Callbacks are problematic, especially over HTTP, so its not really an option because of its complexity. I think that most times you can really work around it, use versions, or we can work on other features to enable it. For example, we can easily provide a full realtime "doc exists" API (with realtime version response).
On Thursday, June 9, 2011 at 11:22 PM, Karussell wrote:
> On Jun 9, 9:21 pm, Shay Banon \<[shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) ([http://elasticsearch.com](http://elasticsearch.com))\> wrote:
>
> > No, there isn't one, and its problematic to provide? It means that elasticsearch will need to keep track of the indexed documents, so you will be able to tell if it happened after them. Parent child visibility can be ensure by indexing the parent doc before the child doc for example.
>
> E.g. FAST ESP provides listeners for this purposes and especially with
> the NRT feature one now cannot give a guarantuee when a doc gets
> indexed.
>
> > It means that elasticsearch will need to keep track of the indexed documents
>
> maybe only for a subset? or one can mark only one document as a
> trigger?
---
### Author:  [@Curtis\_Caravone](https://discuss.elastic.co/u/Curtis_Caravone)
#### Post date: [June 21, 2011, 6:12pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/6 "2011-06-21T18:12:11Z")
I was thinking of something really simple:
- "wait for refresh" call registers a listener
- When the next index refresh finishes, notify (and deregister) all
registered listeners -\> each listener sends an "OK" response
No need to track documents, just keep a set of listeners (maybe you need one
set per index or per shard?)
I thought all the request handling was asynchronous anyway, so the callback
for the HTTP response wouldn't be a problem?
To give a little more detail on my use case:
I'm not really doing simple parent/child relationships, it's more like:
A: Index a set of documents
B: Index a related set of documents into another index
I don't want any of the docs from set B to show up in search results until I
am sure that all the docs from set A are visible to searching as well.
There may not be explicit parent-child relationships to follow at the
document level. All retrieval is done through search queries.
Curtis
On Thu, Jun 9, 2011 at 1:25 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> Callbacks are problematic, especially over HTTP, so its not really an
> option because of its complexity. I think that most times you can really
> work around it, use versions, or we can work on other features to enable it.
> For example, we can easily provide a full realtime "doc exists" API (with
> realtime version response).
>
> On Thursday, June 9, 2011 at 11:22 PM, Karussell wrote:
>
> On Jun 9, 9:21 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> No, there isn't one, and its problematic to provide? It means that
> elasticsearch will need to keep track of the indexed documents, so you will
> be able to tell if it happened after them. Parent child visibility can be
> ensure by indexing the parent doc before the child doc for example.
>
> E.g. FAST ESP provides listeners for this purposes and especially with
> the NRT feature one now cannot give a guarantuee when a doc gets
> indexed.
>
> It means that elasticsearch will need to keep track of the indexed
> documents
>
> maybe only for a subset? or one can mark only one document as a
> trigger?
---
### Author:  [@Remy\_Gendron](https://discuss.elastic.co/u/Remy_Gendron)
#### Post date: [June 22, 2011, 2:44am UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/7 "2011-06-22T02:44:43Z")
We are migrating to es from hibernate search and we intend to use es
not only to support full text searching but, also as our primary mean
of accessing all of our data... Think nosql here. Hence, we are also
indexing the relations between entities so that we can filter on them.
A simple use case made difficult by the NRT nature of es is as follow:
- Use es to load a user's tasks
- Display the tasks in a list view
- Mark one task a completed and update es
- Refresh the list view
The expectation is that the completed task will not show up. I am
willing to block until all indexes and shards involved have been
refreshed.
I totally get the difficulty in implementing such a feature. Until we
have full real time, the following would be great:
- I don't care about changes made by other users. It's OK to have a 1
sec delay before visibility.
- But for actions taken by the user, I need the next page update or
navigation to reflect that.
- So, when a use case requires immediate visibility, the caller will
provide a list of document uid/version pairs along with the query. The
query should block until all the specified documents are visible at
the specified versions (or newer). Then the query executes and
returns.
In fact, I could easily implement this in my service layer but, it
would be great if this was natively supported.
Btw, with the latest nrt work on lucene, do you expect we will get
full rt anytime soon?
Thanks for an incredible product!
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [June 23, 2011, 11:35am UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/8 "2011-06-23T11:35:57Z")
A blocking option, where one sends a bulk request (for example) and it will block until the changes done in the bulk request are made visible by the next refresh is possible. Open an issue.
Lucene refresh cost will get lower in the next Lucene version, but not to a real time level.
On Wednesday, June 22, 2011 at 5:44 AM, Remy Gendron wrote:
> We are migrating to es from hibernate search and we intend to use es
> not only to support full text searching but, also as our primary mean
> of accessing all of our data... Think nosql here. Hence, we are also
> indexing the relations between entities so that we can filter on them.
>
> A simple use case made difficult by the NRT nature of es is as follow:
>
> - Use es to load a user's tasks
> - Display the tasks in a list view
> - Mark one task a completed and update es
> - Refresh the list view
>
> The expectation is that the completed task will not show up. I am
> willing to block until all indexes and shards involved have been
> refreshed.
>
> I totally get the difficulty in implementing such a feature. Until we
> have full real time, the following would be great:
>
> - I don't care about changes made by other users. It's OK to have a 1
> sec delay before visibility.
> - But for actions taken by the user, I need the next page update or
> navigation to reflect that.
> - So, when a use case requires immediate visibility, the caller will
> provide a list of document uid/version pairs along with the query. The
> query should block until all the specified documents are visible at
> the specified versions (or newer). Then the query executes and
> returns.
>
> In fact, I could easily implement this in my service layer but, it
> would be great if this was natively supported.
>
> Btw, with the latest nrt work on lucene, do you expect we will get
> full rt anytime soon?
>
> Thanks for an incredible product!
---
### Author:  [@Curtis\_Caravone](https://discuss.elastic.co/u/Curtis_Caravone)
#### Post date: [June 25, 2011, 2:55pm UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/9 "2011-06-25T14:55:42Z")
I opened an issue:
>
>
> Feature request: Provide an option to the index operation that will wait until …the next scheduled refresh occurs before returning a response. After the response is returned, all documents indexed in that operation should be visible for search.
thanks!
On Thu, Jun 23, 2011 at 4:35 AM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> A blocking option, where one sends a bulk request (for example) and it
> will block until the changes done in the bulk request are made visible by
> the next refresh is possible. Open an issue.
>
> Lucene refresh cost will get lower in the next Lucene version, but not to a
> real time level.
>
> On Wednesday, June 22, 2011 at 5:44 AM, Remy Gendron wrote:
>
> We are migrating to es from hibernate search and we intend to use es
> not only to support full text searching but, also as our primary mean
> of accessing all of our data... Think nosql here. Hence, we are also
> indexing the relations between entities so that we can filter on them.
>
> A simple use case made difficult by the NRT nature of es is as follow:
>
> - Use es to load a user's tasks
> - Display the tasks in a list view
> - Mark one task a completed and update es
> - Refresh the list view
>
> The expectation is that the completed task will not show up. I am
> willing to block until all indexes and shards involved have been
> refreshed.
>
> I totally get the difficulty in implementing such a feature. Until we
> have full real time, the following would be great:
>
> - I don't care about changes made by other users. It's OK to have a 1
> sec delay before visibility.
> - But for actions taken by the user, I need the next page update or
> navigation to reflect that.
> - So, when a use case requires immediate visibility, the caller will
> provide a list of document uid/version pairs along with the query. The
> query should block until all the specified documents are visible at
> the specified versions (or newer). Then the query executes and
> returns.
>
> In fact, I could easily implement this in my service layer but, it
> would be great if this was natively supported.
>
> Btw, with the latest nrt work on lucene, do you expect we will get
> full rt anytime soon?
>
> Thanks for an incredible product!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:02am UTC](https://discuss.elastic.co/t/wait-for-refresh-command/4589/10 "2017-07-06T04:02:36Z")