Bulk index not returning source

Elastic Stack Elasticsearch
1

HI,
is there a way to return the whole _source object when using bulkIndex operation ? And how ? I am using the javascript api if that matters.
Thanks

2

Hi @dbrezoev,

no, this is not possible. Note that the bulk API returns items in the same order as in the request (i.e. the n-th item in the response, corresponds to the n-th item in the request). If you store the items of one bulk locally e.g. in an array you should be able to "merge" the information from the response if you need to.

Daniel

3

Thanks Daniel,

another question just to be sure. So if one of the queries somewhere in the middle fails, the order still remains the same, but in this particular failed item, I have flag with error = true for example? And so I can still trust that the order is the same ?

4

Hi @dbrezoev,

yes, this will exactly work as you'd expect it. Here is a somewhat contrived but complete example which you can use to experiment yourself:

DELETE my_index

POST /my_index/docs/1
{
    "text": "I was here first"
}


POST /_bulk
{ "create" : { "_index": "my_index", "_type": "docs", "_id": "3"} }
{"text": "Third things first."}
{ "create" : { "_index": "my_index", "_type": "docs", "_id": "1"} }
{"text": "I am doomed to fail because a document with this id exists already"}
{ "create" : { "_index": "my_index", "_type": "docs", "_id": "2"} }
{"text": "This will work just fine"}

The first and the last document will be indexed just fine but the middle one will fail because we've already indexed a document with the same id and use the create operation to force that documents will be newly created.

Here is the response that we get, note that all bulk item responses are in the same order as in the request:

{
   "took": 3,
   "errors": true,
   "items": [
      {
         "create": {
            "_index": "my_index",
            "_type": "docs",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "created": true,
            "status": 201
         }
      },
      {
         "create": {
            "_index": "my_index",
            "_type": "docs",
            "_id": "1",
            "status": 409,
            "error": {
               "type": "version_conflict_engine_exception",
               "reason": "[docs][1]: version conflict, document already exists (current version [1])",
               "index_uuid": "cVDMr3SKTM-gmSkqMdB8Hw",
               "shard": "3",
               "index": "my_index"
            }
         }
      },
      {
         "create": {
            "_index": "my_index",
            "_type": "docs",
            "_id": "2",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "created": true,
            "status": 201
         }
      }
   ]
}

Hope that helps.

Daniel

1 Like
5

Everything is clear now.
Thanks again!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.