# How to wait for a CreateIndexRequest to really finish (using java TransportClient) **URL:** \ **Category:** Elasticsearch\ **Created:** [January 8, 2013, 11:00am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261 "2013-01-08T11:00:27Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 8, 2013, 11:00am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/1 "2013-01-08T11:00:27Z")
Hi there, I'm using the native client (TransportClient) and I'm running the following code: Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", "elasticsearch").build(); TransportClient client = new TransportClient(settings); client.addTransportAddress(new InetSocketTransportAddress("localhost", 9300)); Settings indexSettings = ImmutableSettings.settingsBuilder().put("number\_of\_shards", 4) .put("number\_of\_replicas", 3).build(); CreateIndexRequestBuilder createIndexBuilder = client.admin().indices().prepareCreate("newindex"); createIndexBuilder.setSettings(indexSettings); CreateIndexResponse createIndexResponse = createIndexBuilder.execute().actionGet(); System.out.printf("acknowledged: %s%n", createIndexResponse.acknowledged()); GetRequestBuilder get = client.prepareGet(); get.setIndex("newindex"); get.setType("mytype"); get.setId("myid"); try { GetResponse response = get.execute().actionGet(1000); System.out.printf("exists: %s%n", response.exists()); } catch (Throwable ex) { ex.printStackTrace(); } When I execute it for the first time on a fresh node, I always get the following error: acknowledged: true org.elasticsearch.action.NoShardAvailableActionException: [newindex][0] No shard available for [[newindex][mytype][myid]: routing [null]] at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:139) at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:124) at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:71) at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:46) at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:217) at org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:199) ... When I delete the index and run this code again, it runs just fine (and prints exists:false obviously). When I execute the GetRequest a little bit later, it runs just fine too. How do I wait for the CreateIndexRequest to be really completed before starting other processing on the new index? Jaap -- ---
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 8, 2013, 11:09am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/2 "2013-01-08T11:09:48Z")
I've tested it a little bit more, sometimes a new indexname, runs ok, so some chance is involved, but I seem to get it quite often. -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 8, 2013, 11:13am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/3 "2013-01-08T11:13:47Z")
Just add a waitForYellow after index creation. Something like this: [https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32](https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32) [https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32](https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32) It should help. David. Le 8 janvier 2013 à 12:00, Jaap Taal [jaap@q42.nl](mailto:jaap@q42.nl) a écrit : > Hi there, > > I'm using the native client (TransportClient) and I'm running the following > code: > > Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", > "elasticsearch").build(); TransportClient client = new > TransportClient(settings); client.addTransportAddress(new > InetSocketTransportAddress("localhost", 9300)); Settings indexSettings = > ImmutableSettings.settingsBuilder().put("number\_of\_shards", 4) > .put("number\_of\_replicas", 3).build(); CreateIndexRequestBuilder > createIndexBuilder = client.admin().indices().prepareCreate("newindex"); > createIndexBuilder.setSettings(indexSettings); CreateIndexResponse > createIndexResponse = createIndexBuilder.execute().actionGet(); > System.out.printf("acknowledged: %s%n", createIndexResponse.acknowledged()); > GetRequestBuilder get = client.prepareGet(); get.setIndex("newindex"); > get.setType("mytype"); get.setId("myid"); try { GetResponse response = > get.execute().actionGet(1000); System.out.printf("exists: %s%n", > response.exists()); } catch (Throwable ex) { ex.printStackTrace(); } > When I execute it for the first time on a fresh node, I always get the > following error: > acknowledged: true > org.elasticsearch.action.NoShardAvailableActionException: [newindex][0] No > shard available for [[newindex][mytype][myid]: routing [null]] > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:139) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:124) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:71) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:46) > at > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:217) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:199) > ... > > When I delete the index and run this code again, it runs just fine (and > prints exists:false obviously). > When I execute the GetRequest a little bit later, it runs just fine too. > > How do I wait for the CreateIndexRequest to be really completed before > starting other processing on the new index? > > Jaap > > -- -- David Pilato [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 8, 2013, 3:33pm UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/4 "2013-01-08T15:33:50Z")
Nice, that really helped! FYI, I opened the following github issue to ask for some more automation for this case: > > > See the issue on the forum below. > What I would like to suggest is to add a param…eter to the CreateIndexRequest that waits for the shards to become available, or maybe wait for the cluster status to turn yellow. > Like the refresh boolean that can be passed in with an IndexRequest. This way I wouldn't have to work around this behaviour with a separate prepareHealth().setWaitForYellowStatus() > > See also > https://groups.google.com/forum/?fromgroups=#!topic/elasticsearch/uz55XJO9LP4 this is what I used in the end: ClusterHealthRequestBuilder healthRequest = client.admin().cluster().prepareHealth(); _healthRequest.setIndices(newIndexName); // only request health of this index..._ healthRequest.setWaitForYellowStatus(); ClusterHealthResponse healthResponse = healthRequest.execute().actionGet(); System.out.printf("status: %s%n", healthResponse.status()); To avoid having parallel index processes on the same cluster causing the health check for my case to wait longer than needed. Is that going to do what I expect of it? Thanks! On Tuesday, January 8, 2013 12:13:47 PM UTC+1, David Pilato wrote: > Just add a waitForYellow after index creation. > > Something like this: > [https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32](https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32) > > It should help. > > David. > > Le 8 janvier 2013 à 12:00, Jaap Taal \\> a > écrit : > > Hi there, > > I'm using the native client (TransportClient) and I'm running the > following code: > > Settings settings = ImmutableSettings.settingsBuilder().put("cluster.name", > "elasticsearch").build(); TransportClient client = new > TransportClient(settings); client.addTransportAddress(new > InetSocketTransportAddress("localhost", 9300)); Settings indexSettings = > ImmutableSettings.settingsBuilder().put("number\_of\_shards", 4) > .put("number\_of\_replicas", 3).build(); CreateIndexRequestBuilder > createIndexBuilder = client.admin().indices().prepareCreate("newindex"); > createIndexBuilder.setSettings(indexSettings); CreateIndexResponse > createIndexResponse = createIndexBuilder.execute().actionGet(); > System.out.printf("acknowledged: %s%n", > createIndexResponse.acknowledged()); GetRequestBuilder get = > client.prepareGet(); get.setIndex("newindex"); get.setType("mytype"); > get.setId("myid"); try { GetResponse response = > get.execute().actionGet(1000); System.out.printf("exists: %s%n", > response.exists()); } catch (Throwable ex) { ex.printStackTrace(); } > When I execute it for the first time on a fresh node, I always get the > following error: > acknowledged: true > org.elasticsearch.action.NoShardAvailableActionException: [newindex][0] > No shard available for [[newindex][mytype][myid]: routing [null]] > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:139) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:124) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:71) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:46) > at > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:217) > at > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:199) > ... > > When I delete the index and run this code again, it runs just fine (and > prints exists:false obviously). > When I execute the GetRequest a little bit later, it runs just fine too. > > How do I wait for the CreateIndexRequest to be really completed before > starting other processing on the new index? > > Jaap > > -- > > -- > David Pilato > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 9, 2013, 1:04am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/5 "2013-01-09T01:04:42Z")
I don't fully understand your issue. 1. you create an index 2. you issue immediately a get request on it, despite having inserted a single document. This will never work out right. Why don't you insert documents first? What is your intention? Creating an index is an asynchronous node operation executed by the master of the cluster. The cluster master node receives the creation request, submits the index creation to the participating nodes, and returns a response to the client. Meanwhile, depending on how busy the other nodes are, they create the shards, and all shards register in the cluster state for routing. When enough shards have registered, the index state changes. This operation is encapsulated by an index-level block. Unfortunately, Elasticsearch does not exactly know in advance when the first shard will appear in the routing, so, because the create index call must not hang, it returns early. Adding a globally blocked call to the index creation API would introduce other problems. Probably, index creations could hang a client more easily then. Waiting for state yellow/green is similar for waiting for the end of the index being blocked by the master. There is another method in the cluster health request "waitForActiveShards()" which can wait for a certain number of shards in that index. In most cases, waiting for one shard is enough to continue without error. Note, it is a common task to check the cluster health first before indexing (either for yellow or green). ES does not hinder you from indexing (or getting) docs, even if the cluster is red. Jörg -- ---
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 9, 2013, 9:37am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/6 "2013-01-09T09:37:58Z")
Whoops, the last question wasn't really clear. About the health call, setIndices On Tuesday, January 8, 2013 4:33:50 PM UTC+1, Jaap Taal wrote: > Nice, that really helped! > > FYI, I opened the following github issue to ask for some more automation > for this case: > [An acknowledged CreateIndexRequest followed by GetRequest sometimes fails · Issue #2527 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/2527) > > this is what I used in the end: > ClusterHealthRequestBuilder healthRequest = > client.admin().cluster().prepareHealth(); > _healthRequest.setIndices(newIndexName); // only request health of this > index..._ > healthRequest.setWaitForYellowStatus(); > ClusterHealthResponse healthResponse = healthRequest.execute().actionGet(); > System.out.printf("status: %s%n", healthResponse.status()); > > To avoid having parallel index processes on the same cluster causing the > health check for my case to wait longer than needed. > Is that going to do what I expect of it? > > Thanks! > > On Tuesday, January 8, 2013 12:13:47 PM UTC+1, David Pilato wrote: > > > Just add a waitForYellow after index creation. > > > > Something like this: > > [https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32](https://github.com/elasticsearchfr/hands-on/blob/answers/src/test/java/org/elasticsearchfr/handson/StartNode.java#L32) > > > > It should help. > > > > David. > > > > Le 8 janvier 2013 à 12:00, Jaap Taal [ja...@q42.nl](mailto:ja...@q42.nl) a écrit : > > > > Hi there, > > > > I'm using the native client (TransportClient) and I'm running the > > following code: > > > > Settings settings = ImmutableSettings.settingsBuilder().put(" > > cluster.name", "elasticsearch").build(); TransportClient client = new > > TransportClient(settings); client.addTransportAddress(new > > InetSocketTransportAddress("localhost", 9300)); Settings indexSettings = > > ImmutableSettings.settingsBuilder().put("number\_of\_shards", 4) > > .put("number\_of\_replicas", 3).build(); CreateIndexRequestBuilder > > createIndexBuilder = client.admin().indices().prepareCreate("newindex"); > > createIndexBuilder.setSettings(indexSettings); CreateIndexResponse > > createIndexResponse = createIndexBuilder.execute().actionGet(); > > System.out.printf("acknowledged: %s%n", > > createIndexResponse.acknowledged()); GetRequestBuilder get = > > client.prepareGet(); get.setIndex("newindex"); get.setType("mytype"); > > get.setId("myid"); try { GetResponse response = > > get.execute().actionGet(1000); System.out.printf("exists: %s%n", > > response.exists()); } catch (Throwable ex) { ex.printStackTrace(); } > > When I execute it for the first time on a fresh node, I always get the > > following error: > > acknowledged: true > > org.elasticsearch.action.NoShardAvailableActionException: [newindex][0] > > No shard available for [[newindex][mytype][myid]: routing [null]] > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.perform(TransportShardSingleOperationAction.java:139) > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$AsyncSingleAction.start(TransportShardSingleOperationAction.java:124) > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:71) > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction.doExecute(TransportShardSingleOperationAction.java:46) > > at > > org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:217) > > at > > org.elasticsearch.action.support.single.shard.TransportShardSingleOperationAction$TransportHandler.messageReceived(TransportShardSingleOperationAction.java:199) > > ... > > > > When I delete the index and run this code again, it runs just fine (and > > prints exists:false obviously). > > When I execute the GetRequest a little bit later, it runs just fine too. > > > > How do I wait for the CreateIndexRequest to be really completed before > > starting other processing on the new index? > > > > Jaap > > > > -- > > > > -- > > David Pilato > > [http://www.scrutmydocs.org/](http://www.scrutmydocs.org/) > > [http://dev.david.pilato.fr/](http://dev.david.pilato.fr/) > > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs -- ---
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 9, 2013, 9:56am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/7 "2013-01-09T09:56:52Z")
It's a narrowed down example. When you're a building complex indexing-application the situation could arise that you've just created an index in one function and try to check for the existence of a document by id in another. Now the execution flow of the two functions in my case are on the same thread fortunately, so I'd like to make sure that after my index gets created, the other code doesn't fail with the exception that shard number 0 is not ready. The point is that the get by id operation directly routes the key to a shard that is not ready yet. When you're searching this also happens: acknowledged: true org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], total failure; shardFailures {[_na_][prices-2p-d20130118-i20130107][0]: No active shards}{[_na_][prices-2p-d20130118-i20130107][1]: No active shards}{[_na_][prices-2p-d20130118-i20130107][2]: No active shards}{[_na_][prices-2p-d20130118-i20130107][3]: No active shards} at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:260) at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:145) at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:58) at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:48) What I proposed was an optional flag just like "refresh=true", but then for the CreateIndexRequest. If not desirable, this quirk should be documented and an example how to work around it could be suggested by [elasticsearch.org](http://elasticsearch.org). Checking the health of a cluster before indexing each document is not feasible, it would be too slow. On Wednesday, January 9, 2013 2:04:42 AM UTC+1, Jörg Prante wrote: > I don't fully understand your issue. > > 1. you create an index > 2. you issue immediately a get request on it, despite having inserted a > single document. > > This will never work out right. Why don't you insert documents first? What > is your intention? > > Creating an index is an asynchronous node operation executed by the master > of the cluster. The cluster master node receives the creation request, > submits the index creation to the participating nodes, and returns a > response to the client. Meanwhile, depending on how busy the other nodes > are, they create the shards, and all shards register in the cluster state > for routing. When enough shards have registered, the index state changes. > This operation is encapsulated by an index-level block. Unfortunately, > Elasticsearch does not exactly know in advance when the first shard will > appear in the routing, so, because the create index call must not hang, it > returns early. > > Adding a globally blocked call to the index creation API would introduce > other problems. Probably, index creations could hang a client more easily > then. > > Waiting for state yellow/green is similar for waiting for the end of the > index being blocked by the master. > > There is another method in the cluster health request > "waitForActiveShards()" which can wait for a certain number of shards in > that index. In most cases, waiting for one shard is enough to continue > without error. > > Note, it is a common task to check the cluster health first before > indexing (either for yellow or green). ES does not hinder you from indexing > (or getting) docs, even if the cluster is red. > > Jörg -- ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [January 9, 2013, 6:55pm UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/8 "2013-01-09T18:55:32Z")
It's not a quirk. It's the distributed architecture of Elasticsearch. There is no easy method to implement a synchronous index creation, as I tried to clarify. It would introduce more issues than it may solve. Behind the scene, your proposal would be implemented by a sequence of two commands, first index creation, then a cluster state wait, then a response to the client. It's just "sugar coating", no new function, because clients can use this two command sequence already. Note, a health check returns immediately, or waits 200ms internally in a loop until the requested condition is satisfied. I can hardly imagine of a faster method in a distributed system, where the cluster state propagates through a number of connected nodes. A parameter name proposal of "refresh=true" is rather unfortunate. It shouldn't be confused with an index refresh since there are no documents to refresh. Some suggestions: - use index aliases for shard presence guarantee. If you can create a variable number of indices to search in parallel, adding a new index to existing indices by an index alias will never throw a message "no active shards". - use front-end index control, for example by a reverse proxy. If malevolent clients ask for documents which might not be there, catch the index creation error message, and wait in the front-end for cluster health, and execute the search afterwards. Jörg On Wednesday, January 9, 2013 10:56:52 AM UTC+1, Jaap Taal wrote: > What I proposed was an optional flag just like "refresh=true", but then > for the CreateIndexRequest. If not desirable, this quirk should be > documented and an example how to work around it could be suggested by > [elasticsearch.org](http://elasticsearch.org). > > Checking the health of a cluster before indexing each document is not > feasible, it would be too slow. -- ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 9, 2013, 7:36pm UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/9 "2013-01-09T19:36:25Z")
I love the first suggestion. Smart. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 9 janv. 2013 à 19:55, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) a écrit : It's not a quirk. It's the distributed architecture of Elasticsearch. There is no easy method to implement a synchronous index creation, as I tried to clarify. It would introduce more issues than it may solve. Behind the scene, your proposal would be implemented by a sequence of two commands, first index creation, then a cluster state wait, then a response to the client. It's just "sugar coating", no new function, because clients can use this two command sequence already. Note, a health check returns immediately, or waits 200ms internally in a loop until the requested condition is satisfied. I can hardly imagine of a faster method in a distributed system, where the cluster state propagates through a number of connected nodes. A parameter name proposal of "refresh=true" is rather unfortunate. It shouldn't be confused with an index refresh since there are no documents to refresh. Some suggestions: - use index aliases for shard presence guarantee. If you can create a variable number of indices to search in parallel, adding a new index to existing indices by an index alias will never throw a message "no active shards". - use front-end index control, for example by a reverse proxy. If malevolent clients ask for documents which might not be there, catch the index creation error message, and wait in the front-end for cluster health, and execute the search afterwards. Jörg On Wednesday, January 9, 2013 10:56:52 AM UTC+1, Jaap Taal wrote: > What I proposed was an optional flag just like "refresh=true", but then for the CreateIndexRequest. If not desirable, this quirk should be documented and an example how to work around it could be suggested by [elasticsearch.org](http://elasticsearch.org). > > Checking the health of a cluster before indexing each document is not feasible, it would be too slow. -- -- ---
**Author:** ![q42jaap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/q42jaap/32/2241_2.png) [@q42jaap](https://discuss.elastic.co/u/q42jaap)\ **Post date:** [January 9, 2013, 9:55pm UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/10 "2013-01-09T21:55:11Z")
Understood, will not nag again 😉 Thanks for you explanation! Using an alias is definitely what I want, and since a newly created index is always empty it would allow me to do this without waiting for a status. Though it's always good to know how and why you should wait for active shards or the yellow status (I'd think the active shards are a better method for a newly created index). Jaap Taal [Q42 BV | tel 070 44523 42 | direct 070 44523 65 | [http://q42.nl](http://q42.nl) | Waldorpstraat 17F, Den Haag | KvK 30164662 ] On Wed, Jan 9, 2013 at 7:55 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > It's not a quirk. It's the distributed architecture of Elasticsearch. > There is no easy method to implement a synchronous index creation, as I > tried to clarify. It would introduce more issues than it may solve. > > Behind the scene, your proposal would be implemented by a sequence of two > commands, first index creation, then a cluster state wait, then a response > to the client. It's just "sugar coating", no new function, because clients > can use this two command sequence already. Note, a health check returns > immediately, or waits 200ms internally in a loop until the requested > condition is satisfied. I can hardly imagine of a faster method in a > distributed system, where the cluster state propagates through a number of > connected nodes. > > A parameter name proposal of "refresh=true" is rather unfortunate. It > shouldn't be confused with an index refresh since there are no documents to > refresh. > > Some suggestions: > > - use index aliases for shard presence guarantee. If you can create a > variable number of indices to search in parallel, adding a new index to > existing indices by an index alias will never throw a message "no active > shards". > > - use front-end index control, for example by a reverse proxy. If > malevolent clients ask for documents which might not be there, catch the > index creation error message, and wait in the front-end for cluster health, > and execute the search afterwards. > > Jörg > > On Wednesday, January 9, 2013 10:56:52 AM UTC+1, Jaap Taal wrote: > > > What I proposed was an optional flag just like "refresh=true", but then > > for the CreateIndexRequest. If not desirable, this quirk should be > > documented and an example how to work around it could be suggested by > > [elasticsearch.org](http://elasticsearch.org). > > > > Checking the health of a cluster before indexing each document is not > > feasible, it would be too slow. > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:57am UTC](https://discuss.elastic.co/t/how-to-wait-for-a-createindexrequest-to-really-finish-using-java-transportclient/10261/11 "2017-07-06T02:57:06Z")