# Client refresh in unit tests not always fast enough **URL:** \ **Category:** Elasticsearch\ **Created:** [October 13, 2010, 4:51pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438 "2010-10-13T16:51:18Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![John\_Chang](https://avatars.discourse-cdn.com/v4/letter/j/ecc23a/32.png) [@John\_Chang](https://discuss.elastic.co/u/John_Chang)\ **Post date:** [October 13, 2010, 4:51pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/1 "2010-10-13T16:51:18Z")
I am running unit tests with a local cluster. They are generally very reliable, but every once in a while one of them fails because a test (A) indexes something, (B) then calls refresh for the client, (C) and then tries to look up the recently indexed item, and it is not there. The same test will work 95%+ of the time. My guess is that the refresh is not always fast enough (I read it is "near" real-time). Is there anything I can do to make these tests more reliable? I could add a little sleep time after the refresh, but that seems like it would be an unreliable kludge (not to mention would slow down my tests a lot). HOW I REFRESH: elasticSearchClient.admin().indices().refresh(Requests.refreshRequest()).actionGet(); HOW I SET UP THE LOCAL TEST CLUSTER: hostNode = NodeBuilder.nodeBuilder().loadConfigSettings(false).clusterName("test.cluster").local(true).settings( ImmutableSettings.settingsBuilder() .put("index.number\_of\_shards", 1) .put("index.number\_of\_replicas", 1) .build() ``` ).node().start(); elasticSearchClient = hostNode.client(); ``` ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 13, 2010, 6:39pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/2 "2010-10-13T18:39:03Z")
The refresh API, once returned (successfully), guarantees that anything that happened before it was executed will become searchable. The near real time aspect comes from the fact that its not called on every operations, just periodically (internally). Is there a chance for a concurrency glitch here (are A, B, and C executed serially)? or maybe the indices have not recovered yet? -shay.banon On Wed, Oct 13, 2010 at 6:51 PM, John Chang [jchangkihtest2@gmail.com](mailto:jchangkihtest2@gmail.com)wrote: > I am running unit tests with a local cluster. They are generally very > reliable, but every once in a while one of them fails because a test (A) > indexes something, (B) then calls refresh for the client, (C) and then > tries > to look up the recently indexed item, and it is not there. The same test > will work 95%+ of the time. My guess is that the refresh is not always > fast > enough (I read it is "near" real-time). Is there anything I can do to make > these tests more reliable? I could add a little sleep time after the > refresh, but that seems like it would be an unreliable kludge (not to > mention would slow down my tests a lot). > > HOW I REFRESH: > > elasticSearchClient.admin().indices().refresh(Requests.refreshRequest()).actionGet(); > > HOW I SET UP THE LOCAL TEST CLUSTER: > hostNode = > > NodeBuilder.nodeBuilder().loadConfigSettings(false).clusterName("test.cluster").local(true).settings( > ImmutableSettings.settingsBuilder() > .put("index.number\_of\_shards", 1) > .put("index.number\_of\_replicas", 1) > .build() > > ``` > ).node().start(); > elasticSearchClient = hostNode.client(); > > ``` > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Client-refresh-in-unit-tests-not-always-fast-enough-tp1695708p1695708.html](http://elasticsearch-users.115913.n3.nabble.com/Client-refresh-in-unit-tests-not-always-fast-enough-tp1695708p1695708.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![John\_Chang](https://avatars.discourse-cdn.com/v4/letter/j/ecc23a/32.png) [@John\_Chang](https://discuss.elastic.co/u/John_Chang)\ **Post date:** [October 13, 2010, 7:58pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/3 "2010-10-13T19:58:04Z")
A, B, and C are called serially in one single thread. As for recovery, I don't think that is the issue, as I am not trying to search data that was indexed before program startup; I am only searching docs that were indexed with that client same instance. I'm not relying on any persistent data from before the test ran, if that is what you mean by index recovery. I was not checking for successful invocation of the refresh, so I tried the below method for refreshing. However, I did still see one intermittent failure with the below refresh. ``` private void refresh() { int maxTries = 3; for (int i = 0; i < maxTries; i++) { BroadcastOperationResponse response = elasticSearchClient.admin().indices().refresh(Requests.refreshRequest()).actionGet(); if (response.failedShards() > 0) { System.out.println("did not refresh index search client successfully; retry"); try { Thread.sleep(1000); } catch (InterruptedException e) { e.printStackTrace(); } } else { return; } } Assert.fail("unable to refresh index search client."); } ``` ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 13, 2010, 8:00pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/4 "2010-10-13T20:00:01Z")
Can you extract the test to a standalone one and I will run it? Have no problem running it continuously until it fails... On Wed, Oct 13, 2010 at 9:58 PM, John Chang [jchangkihtest2@gmail.com](mailto:jchangkihtest2@gmail.com)wrote: > A, B, and C are called serially in one single thread. > > As for recovery, I don't think that is the issue, as I am not trying to > search data that was indexed before program startup; I am only searching > docs that were indexed with that client same instance. I'm not relying on > any persistent data from before the test ran, if that is what you mean by > index recovery. > > I was not checking for successful invocation of the refresh, so I tried the > below method for refreshing. However, I did still see one intermittent > failure with the below refresh. > > private void refresh() { > int maxTries = 3; > for (int i = 0; i \< maxTries; i++) { > BroadcastOperationResponse response = > > elasticSearchClient.admin().indices().refresh(Requests.refreshRequest()).actionGet(); > if (response.failedShards() \> 0) { > System.out.println("did not refresh index search client > successfully; retry"); > try { > Thread.sleep(1000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } else { > return; > } > } > > ``` > Assert.fail("unable to refresh index search client."); > > ``` > > ## } > > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Client-refresh-in-unit-tests-not-always-fast-enough-tp1695708p1696864.html](http://elasticsearch-users.115913.n3.nabble.com/Client-refresh-in-unit-tests-not-always-fast-enough-tp1695708p1696864.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![thalej](https://avatars.discourse-cdn.com/v4/letter/t/6bbea6/32.png) [@thalej](https://discuss.elastic.co/u/thalej)\ **Post date:** [February 11, 2014, 8:26pm UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/5 "2014-02-11T20:26:44Z")
Hello - I am seeing the same issue as reported here (I am running ES 0.90.10). Does anyone have suggestions for resolving this? Thank you. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:50am UTC](https://discuss.elastic.co/t/client-refresh-in-unit-tests-not-always-fast-enough/3438/6 "2017-07-06T01:50:56Z")