# Bulk update caused DocumentMissing after Bulk Insert **URL:** \ **Category:** Elasticsearch\ **Created:** [April 15, 2019, 2:31am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808 "2019-04-15T02:31:30Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Zt\_Zeng](https://avatars.discourse-cdn.com/v4/letter/z/b77776/32.png) [@Zt\_Zeng](https://discuss.elastic.co/u/Zt_Zeng)\ **Post date:** [April 15, 2019, 2:31am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/1 "2019-04-15T02:31:30Z")
I am working on a [tool](https://github.com/zzt93/syncer) which sync change from MySQL to ES. When I am testing the tool, I find some thing that can't understand. Then following is some log: ``` # bulk insert about 100 docs 2019-04-14 11:16:26,049 index {[test_0][news][36], source[...]}, index {..} # bulk update 100 docs by id 2019-04-14 11:16:26,428 update {[test_0][news][36], doc[index {[null][null][null], source[{...}], detect_noop[true]} # some docs update report DocumentMissing item.getFailure().getCause() instanceof DocumentMissingException ``` So any reason why this happens? (I am using ES 5.6) ---
**Author:** ![Zt\_Zeng](https://avatars.discourse-cdn.com/v4/letter/z/b77776/32.png) [@Zt\_Zeng](https://discuss.elastic.co/u/Zt_Zeng)\ **Post date:** [April 24, 2019, 11:28pm UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/2 "2019-04-24T23:28:56Z")
Any one here? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 25, 2019, 6:09am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/3 "2019-04-25T06:09:32Z")
Can you share the details of the API calls that your tool is making and the JSON responses? ---
**Author:** ![Zt\_Zeng](https://avatars.discourse-cdn.com/v4/letter/z/b77776/32.png) [@Zt\_Zeng](https://discuss.elastic.co/u/Zt_Zeng)\ **Post date:** [April 27, 2019, 3:23am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/4 "2019-04-27T03:23:36Z")
the request API: ``` BulkRequestBuilder bulkRequest = client.prepareBulk(); for (SyncWrapper requestWrapper : aim) { WriteRequest request = requestWrapper.getData(); if (request instanceof IndexRequest) { bulkRequest.add((IndexRequest) request); } else if (request instanceof UpdateRequest) { bulkRequest.add(((UpdateRequest) request)); } else if (request instanceof DeleteRequest) { bulkRequest.add(((DeleteRequest) request)); } } ListenableActionFuture future = bulkRequest.execute(); ``` the response API: ``` BulkResponse bulkResponse = future.get(); if (bulkResponse.hasFailures()) { BulkItemResponse[] items = bulkResponse.getItems(); for (item in items) { if (item.getFailure().getCause() instanceof DocumentMissingException) {} } } ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 27, 2019, 5:05am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/5 "2019-04-27T05:05:26Z")
I tried but failed to reproduce this on 5.6.16. I made `index.json` containing 100 requests: ```auto {"index":{"_type":"news","_id":"1","_index":"test_0"}} {"foo":"bar"} {"index":{"_type":"news","_id":"2","_index":"test_0"}} {"foo":"bar"} ... {"index":{"_type":"news","_id":"100","_index":"test_0"}} {"foo":"bar"} ``` I made `update.json` also containing 100 requests: ```auto {"update":{"_type":"news","_id":"1","_index":"test_0"}} {"doc":{"foo":"baz"}} {"update":{"_type":"news","_id":"2","_index":"test_0"}} {"doc":{"foo":"baz"}} ... {"update":{"_type":"news","_id":"100","_index":"test_0"}} {"doc":{"foo":"baz"}} ``` I created a new index and immediately ran these bulk requests and saw no errors: ```auto $ curl -XDELETE 'http://localhost:9200/test_0?pretty'; curl -XPUT 'http://localhost:9200/test_0?pretty' -H 'Content-type: application/json' --data-binary $'{"settings":{"number_of_shards":2,"number_of_replicas":1}}'; curl 'http://localhost:9200/_bulk?pretty&filter_path=errors' -H 'Content-type: application/x-ndjson' --data-binary @index.json; curl 'http://localhost:9200/_bulk?pretty&filter_path=errors' -H 'Content-type: application/x-ndjson' --data-binary @update.json { "acknowledged" : true, "shards_acknowledged" : true, "index" : "test_0" } { "errors" : false } { "errors" : false } ``` Can you describe how to reproduce the problem you're seeing like this? ---
**Author:** ![Zt\_Zeng](https://avatars.discourse-cdn.com/v4/letter/z/b77776/32.png) [@Zt\_Zeng](https://discuss.elastic.co/u/Zt_Zeng)\ **Post date:** [April 27, 2019, 5:10am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/6 "2019-04-27T05:10:31Z")
I think I found the error, the index request the update request is made by two thread, so even though index request is a little bit faster, we can't ensure the order between those two batch. Thanks for your help. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 27, 2019, 5:17am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/7 "2019-04-27T05:17:42Z")
Ah yes that'd explain it. You have to wait for the response from the indexing request before doing the update, otherwise it's not really "after". ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 25, 2019, 5:17am UTC](https://discuss.elastic.co/t/bulk-update-caused-documentmissing-after-bulk-insert/176808/8 "2019-05-25T05:17:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.