# Java client: typeExists() returns false after successful bulk index - why? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 10, 2014, 2:17am UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189 "2014-01-10T02:17:23Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Nikita\_Tovstoles](https://avatars.discourse-cdn.com/v4/letter/n/f17d59/32.png) [@Nikita\_Tovstoles](https://discuss.elastic.co/u/Nikita_Tovstoles)\ **Post date:** [January 10, 2014, 2:17am UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189/1 "2014-01-10T02:17:23Z")
ES newbie here. i noticed that typeExists() queries return false after successful bulk index() but don't understand why. is that expected? Using java client v 0.90.9. thanks in advance! non-bulk works: @Test public void testTypeExists() { assertFalse(admin().indices().prepareTypesExists(INDEX\_NAME).setTypes("foo").get().isExists()); ``` assertEquals("foo", ``` client().prepareIndex().setIndex(INDEX\_NAME).setType("foo").setId("1").setSource("{"a":"b"}").get() .getType()); assertTrue(admin().indices().prepareTypesExists(INDEX\_NAME).setTypes("foo").get().isExists()); //RETURNS TRUE AS EXPECTED } bulk fails: ``` @Test public void testTypeExistsAfterBulkIndex() { ``` assertFalse(admin().indices().prepareTypesExists(INDEX\_NAME).setTypes("foo").get().isExists()); ``` assertEquals("foo", ``` client().prepareBulk().add(client().prepareIndex().setIndex(INDEX\_NAME).setType("foo").setId("1") .setSource("{"a":"b"}")).execute().actionGet().getItems()[0].getType()); //SUCCEEDS assertTrue(admin().indices().prepareTypesExists(INDEX\_NAME).setTypes("foo").get().isExists()); //FAILS } -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/10d8a7c9-7d4b-4065-a8e5-624fd2750393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/10d8a7c9-7d4b-4065-a8e5-624fd2750393%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [January 10, 2014, 4:13am UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189/2 "2014-01-10T04:13:22Z")
A quick guess: The first one works because the first document for that type is indexed and therefore the type is created when the operation returns. But the second one doesn't work because there is a refresh interval between the completion of a bulk load operation and the actual document being added. And since it's the first document in the type, the type won't exist until that first document is indexed. Which is likely exactly what you want: Bulk operations need to defer until they are processed to allow for optimizations. I don't know Lucene internals, but a B+Tree loads vastly quicker when keys are presorted in bulk instead of added and committed one by one. The experts can chime in later, and if I'm wrong or off base anywhere I welcome the correction! Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6f3374ff-b623-47ca-9e93-3eb2630b6ef3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6f3374ff-b623-47ca-9e93-3eb2630b6ef3%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Nikita\_Tovstoles](https://avatars.discourse-cdn.com/v4/letter/n/f17d59/32.png) [@Nikita\_Tovstoles](https://discuss.elastic.co/u/Nikita_Tovstoles)\ **Post date:** [January 10, 2014, 5:01am UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189/3 "2014-01-10T05:01:33Z")
yes! adding a sleep after Future.get() of bulk op 'fixed' my test - thank you. what you said re: bulk op was submitted but not processed makes sense (perhaps there is a separate API to query for op's completion status?) but what is puzzling to me is that comments in source of _BulkResponse_ seem to imply it is constructed _after_ op completes: Holding a response for each item responding (in order) of the - bulk requests. Each item holds the index/type/id is operated on, and if it failed or not (with the - failure message). ..thus I was expecting that by the time \*ListenableActionFuture.get() \*returns the op is actually completed (not just submitted). Otherwise status properties in embedded BulkItemResponse would not be useful, right? On Thu, Jan 9, 2014 at 8:13 PM, InquiringMind [brian.from.fl@gmail.com](mailto:brian.from.fl@gmail.com)wrote: > A quick guess: The first one works because the first document for that > type is indexed and therefore the type is created when the operation > returns. > > But the second one doesn't work because there is a refresh interval > between the completion of a bulk load operation and the actual document > being added. And since it's the first document in the type, the type won't > exist until that first document is indexed. Which is likely exactly what > you want: Bulk operations need to defer until they are processed to allow > for optimizations. I don't know Lucene internals, but a B+Tree loads vastly > quicker when keys are presorted in bulk instead of added and committed one > by one. > > The experts can chime in later, and if I'm wrong or off base anywhere I > welcome the correction! > > Brian > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/V1A1HbJFio4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/V1A1HbJFio4/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6f3374ff-b623-47ca-9e93-3eb2630b6ef3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6f3374ff-b623-47ca-9e93-3eb2630b6ef3%40googlegroups.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJwaA22WFt%2B%2BQ2z%3DvBMpL\_8ChjBB19OQcDWtTzwy8bd5xQ1sjw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJwaA22WFt%2B%2BQ2z%3DvBMpL_8ChjBB19OQcDWtTzwy8bd5xQ1sjw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [January 10, 2014, 4:51pm UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189/4 "2014-01-10T16:51:04Z")
Bulk requests are processed together, so my only guess is that it returns when it knows enough about whether the operation on each document succeeded or failed. But there may still be a little more work before it finally makes it to the database. Instead of the sleep, try an index refresh. Maybe that will make the test case more deterministic. Also note that even in the first case, there will be a slight delay between the return of the successful update and the ability to query on one or more of the indexed fields. Indexing takes a little time (configurable; typically about 1s or 2s for non-bulk updates), but the index operation returns as soon as it knows all it needs to know about whether the document can be successfully updated. However, a get-by-id operation can be done immediately after an update; only the indexing of the fields as reflected in the on-disk Lucene shard is not quiite realtime and synchronous to the update request. Also see: [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index\_.html#index-refresh](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-refresh) But in general: Whenever I index (create or update) a document, I leave the recommended defaults in force, and only depend on a get-by-id to succeed immediately after. And for bulk loading, I'm happy enough that 97 million documents (not too complex, but still very useful) can be bulk loaded and reloaded on my laptop in under 2.5 hours, and that chunks of "daily" updates of a1 million updates and 1 million deletes can be bulk loaded in 10 or 15 minutes. And I find that ElasticSearch provides a very good balance between optimal update performance and quick search availability of those updates. Also, my experience with your particular test case is very limited. I've recently locked down ElasticSearch and all my mappings so that neither the index nor the type is automatically created nor are unknown fields able to be indexed. I must explicitly create an index and load all of the mappings for all of the fields in all of the types before any documents are indexed. But I digress... Hope this helps! Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/29a731f6-87d9-4c97-9db6-d606f099e947%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/29a731f6-87d9-4c97-9db6-d606f099e947%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:57am UTC](https://discuss.elastic.co/t/java-client-typeexists-returns-false-after-successful-bulk-index-why/15189/5 "2017-07-06T01:57:23Z")